Image encoding/decoding method and recording medium therefor

ABSTRACT

The present invention relates to a method for encoding/decoding a video. To this end, the method for decoding a video may include: generating a merge candidate list of a current block including at least one merge candidate corresponding to each of a plurality of reference picture lists; determining at least one piece of motion information by using the merge candidate list; and generating a prediction block of the current block by using the determined at least one piece of motion information.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application is a Continuation application of U.S. patentapplication Ser. No. 16/301,318 filed on Nov. 13, 2018, which is a U.S.National Stage Application of International Application No.PCT/KR2017/007481, filed on Jul. 12, 2017, which claims the benefitunder 35 USC 119(a) and 365(b) of Korean Patent Application No.10-2016-0088250, filed on Jul. 12, 2016, in the Korean IntellectualProperty Office.

TECHNICAL FIELD

The present invention relates to a method and apparatus forencoding/decoding a video. More particularly, the present inventionrelates to a method and apparatus for performing motion compensation byusing a merge mode.

BACKGROUND ART

Recently, demands for high-resolution and high-quality images such ashigh definition (HD) images and ultra high definition (UHD) images, haveincreased in various application fields. However, higher resolution andquality image data has increasing amounts of data in comparison withconventional image data. Therefore, when transmitting image data byusing a medium such as conventional wired and wireless broadbandnetworks, or when storing image data by using a conventional storagemedium, costs of transmitting and storing increase. In order to solvethese problems occurring with an increase in resolution and quality ofimage data, high-efficiency image encoding/decoding techniques arerequired for higher-resolution and higher-quality images.

Image compression technology includes various techniques, including: aninter-prediction technique of predicting a pixel value included in acurrent picture from a previous or subsequent picture of the currentpicture; an intra-prediction technique of predicting a pixel valueincluded in a current picture by using pixel information in the currentpicture; a transform and quantization technique for compressing energyof a residual signal; an entropy encoding technique of assigning a shortcode to a value with a high appearance frequency and assigning a longcode to a value with a low appearance frequency; etc. Image data may beeffectively compressed by using such image compression technology, andmay be transmitted or stored.

In motion compensation using a conventional merge mode, only a spatialmerge candidate, a temporal merge candidate, a bi-prediction mergecandidate, and a zero merge candidate are added to a merge candidatelist to be used. Accordingly, only uni-directional prediction andbi-directional prediction are used, and thus there is a limit to enhanceencoding efficiency.

In motion compensation using the conventional merge mode, there is alimit in throughput of the merge mode due to dependency between atemporal merge candidate derivation process and a bi-prediction mergecandidate derivation process. Also, the merge candidate derivationprocesses may not be performed in parallel.

In motion compensation using the conventional merge mode, thebi-prediction merge candidate generated through the bi-prediction mergecandidate derivation process is used as motion information. Thus, memoryaccess bandwidth increases during motion compensation, compared to theuni-prediction merge candidate.

In motion compensation using the conventional merge mode, zero mergecandidate derivation is differently performed according to a slice type,and thus hardware logic is complex. Also, a bi-prediction zero mergecandidate is generated through a bi-prediction zero merge candidatederivation process to be used in motion compensation, and thus memoryaccess bandwidth increases.

DISCLOSURE Technical Problem

An object of the present invention is to provide a method and apparatusfor performing motion compensation by using a combined merge candidateto enhance encoding/decoding efficiency of a video.

Another object of the present invention is to provide a method andapparatus for performing motion compensation by using uni-directionprediction, bi-directional prediction, tri-directional prediction, andquad-directional prediction to enhance encoding/decoding efficiency of avideo.

Another object of the present invention is to provide a method andapparatus for determining motion information through parallelization ofthe merge candidate derivation processes, removal of dependency betweenthe merge candidate derivation processes, bi-prediction merge candidatepartitioning, and uni-prediction zero merge candidate derivation so asto increase throughput of the merge mode and to simplify hardware logic.

Technical Solution

A method for decoding a video according to the present inventionincludes: generating a merge candidate list of a current block includingat least one merge candidate corresponding to each of a plurality ofreference picture lists; determining at least one piece of motioninformation by using the merge candidate list; and generating aprediction block of the current block by using the determined at leastone piece of motion information.

In the method for decoding a video, the merge candidate list may includeat least one of a spatial merge candidate derived from a spatialneighbor block of the current block, a temporal merge candidate derivedfrom a collocated block of the current block, a modified spatial mergecandidate derived by modifying the spatial merge candidate, a modifiedtemporal merge candidate derived by modifying the temporal mergecandidate, and a merge candidate having a predefined motion informationvalue.

In the method for decoding a video, the merge candidate list may furtherinclude a combined merge candidate derived by using at least twoselected from a group consisting of the spatial merge candidate, thetemporal merge candidate, the modified spatial merge candidate, and themodified temporal merge candidate.

In the method for decoding a video, the spatial merge candidate may bederived from a sub-block of a neighbor block adjacent to the currentblock, and the temporal merge candidate may be derived from a sub-blockof the collocated block of the current block.

In the method for decoding a video, the generating of the predictionblock of the current block by using the determined at least one piece ofmotion information may include: generating a plurality of temporaryprediction blocks according to an inter-prediction indicator of thecurrent block; and generating the prediction block of the current blockby applying at least one of a weighting factor and an offset to thegenerated plurality of temporary prediction blocks.

In the method for decoding a video, at least one of the weighting factorand the offset may be shared in blocks that are smaller than apredetermined block in size or are deeper than the predetermined blockin depth.

In the method for decoding a video, the merge candidate list may beshared in blocks that are smaller than a predetermined block in size orare deeper than the predetermined block in depth.

In the method for decoding a video, when the current block is smallerthan a predetermined block in size or is deeper than the predeterminedblock in depth, the merge candidate list may be generated based on ahigher block of the current block, the higher block being equal to thepredetermined block in size or in depth.

A method for encoding a video according to the present inventionincludes: generating a merge candidate list of a current block includingat least one merge candidate corresponding to each of a plurality ofreference picture lists; determining at least one piece of motioninformation by using the merge candidate list; and generating aprediction block of the current block by using the determined at leastone piece of motion information.

In the method for encoding a video, the merge candidate list may includeat least one of a spatial merge candidate derived from a spatialneighbor block of the current block, a temporal merge candidate derivedfrom a collocated block of the current block, a modified spatial mergecandidate derived by modifying the spatial merge candidate, a modifiedtemporal merge candidate derived by modifying the temporal mergecandidate, and a merge candidate having a predetermined motioninformation value.

In the method for encoding a video, the merge candidate list may furtherinclude a combined merge candidate derived by using at least twoselected from a group consisting of the spatial merge candidate, thetemporal merge candidate, the modified spatial merge candidate, and themodified temporal merge candidate.

In the method for encoding a video, the spatial merge candidate may bederived from a sub-block of a neighbor block adjacent to the currentblock, and the temporal merge candidate may be derived from a sub-blockof the collocated block of the current block.

In the method for encoding a video, the generating of the predictionblock of the current block by using the determined at least one piece ofmotion information may include: generating a plurality of temporaryprediction blocks according to an inter-prediction indicator of thecurrent block; and generating the prediction block of the current blockby applying at least one of a weighting factor and an offset to thegenerated plurality of temporary prediction blocks.

In the method for encoding a video, at least one of the weighting factorand the offset may be shared in blocks that are smaller than apredetermined block in size or are deeper than the predetermined blockin depth.

In the method for encoding a video, the merge candidate list may beshared in blocks that are smaller than a predetermined block in size orare deeper than the predetermined block in depth.

In the method for encoding a video, when the current block is smallerthan a predetermined block in size or is deeper than the predeterminedblock in depth, the merge candidate list may be generated based on ahigher block of the current block, the higher block being equal to thepredetermined block in size or in depth.

An apparatus for decoding a video according to the present inventionincludes an inter prediction unit that generates a merge candidate listof a current block including at least one merge candidate correspondingto each of a plurality of reference picture lists, determines at leastone piece of motion information by using the merge candidate list, andgenerates a prediction block of the current block by using thedetermined at least one piece of motion information.

An apparatus for encoding a video according to the present inventionincludes an inter prediction unit that generates a merge candidate listof a current block including at least one merge candidate correspondingto each of a plurality of reference picture lists, determines at leastone piece of motion information by using the merge candidate list, andgenerates a prediction block of the current block by using thedetermined at least one piece of motion information.

A readable medium according to the present invention stores a bitstreamformed by a method for encoding a video, the method including:generating a merge candidate list of a current block including at leastone merge candidate corresponding to each of a plurality of referencepicture lists; determining at least one piece of motion information byusing the merge candidate list; and generating a prediction block of thecurrent block by using the determined at least one piece of motioninformation.

Advantageous Effects

In the present invention, provided is a method and apparatus forperforming motion compensation by using a combined merge candidate toenhance encoding/decoding efficiency of a video.

In the present invention, provided is a method and apparatus forperforming motion compensation by using uni-directional prediction,bi-directional prediction, tri-directional prediction, andquad-directional prediction to enhance encoding/decoding efficiency of avideo.

In the present invention, provided is a method and apparatus forperforming motion compensation through parallelization of mergecandidate derivation processes, removal of dependency between the mergecandidate derivation processes, bi-prediction merge candidatepartitioning, and uni-prediction zero merge candidate derivation so asto increase throughput of a merge mode and to simplify hardware logic.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing configurations of an encodingapparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing configurations of a decoding apparatusaccording to an embodiment of the present invention.

FIG. 3 is a view schematically showing a partition structure of an imagewhen encoding and decoding the image.

FIG. 4 is a view showing forms of a prediction unit (PU) that may beincluded in a coding unit (CU).

FIG. 5 is a view showing forms of a transform unit (TU) that may beincluded in a coding unit (CU).

FIG. 6 is a view for explaining an embodiment of a process of intraprediction.

FIG. 7 is a view for explaining an embodiment of a process of interprediction.

FIG. 8 is a view for explaining transform sets according tointra-prediction modes.

FIG. 9 is a view for explaining a process of transform.

FIG. 10 is a view for explaining scanning of quantized transformcoefficients.

FIG. 11 is a view for explaining block partition.

FIG. 12 is a flowchart showing a method for encoding a video by using amerge mode according to the present invention.

FIG. 13 is a flowchart showing a method for decoding a video by using amerge mode according to the present invention.

FIG. 14 is a view showing an example of deriving a spatial mergecandidate of a current block.

FIG. 15 is a view showing an example of adding a spatial merge candidateto a merge candidate list.

FIG. 16 is a view showing an embodiment of deriving and sharing aspatial merge candidate in a CTU.

FIG. 17 is a view showing an example of deriving a temporal mergecandidate of a current block.

FIG. 18 is a view showing an example of adding a temporal mergecandidate to a merge candidate list.

FIG. 19 is a view showing an example of scaling a motion vector of acollocated block to derive a temporal merge candidate of a currentblock.

FIG. 20 is a view showing combined indexes.

FIG. 21 a and FIG. 21 b are views showing an embodiment of a method ofderiving a combined merge candidate.

FIGS. 22 and 23 are views showing an embodiment of deriving a combinedmerge candidate by using at least one of a spatial merge candidate, atemporal merge candidate, and a zero merge candidate, and of adding thecombined merge candidate to a merge candidate list.

FIG. 24 is a view showing an advantage of deriving a combined mergecandidate by using only spatial merge candidates in motion compensationusing a merge mode.

FIG. 25 is a view showing an embodiment of a method of partitioning acombined bi-predictive merge candidate.

FIG. 26 is a view showing an embodiment of a method of deriving a zeromerge candidate.

FIG. 27 is a view showing an embodiment of adding the derived zero mergecandidate to a merge candidate list.

FIG. 28 is a view showing another embodiment of a method of deriving azero merge candidate.

FIG. 29 is a view showing an embodiment of deriving and sharing a mergecandidate list in a CTU.

FIGS. 30 and 31 are views showing examples of syntax of information onmotion compensation.

FIG. 32 is a view showing an embodiment of using a merge mode in blocks,which are smaller than a predetermined block in size, of a CTU.

FIG. 33 is a view showing a method for decoding a video according to thepresent invention.

FIG. 34 is a view showing a method for encoding a video according to thepresent invention.

MODE FOR INVENTION

A variety of modifications may be made to the present invention andthere are various embodiments of the present invention, examples ofwhich will now be provided with reference to drawings and described indetail. However, the present invention is not limited thereto, althoughthe exemplary embodiments can be construed as including allmodifications, equivalents, or substitutes in a technical concept and atechnical scope of the present invention. The similar reference numeralsrefer to the same or similar functions in various aspects. In thedrawings, the shapes and dimensions of elements may be exaggerated forclarity. In the following detailed description of the present invention,references are made to the accompanying drawings that show, by way ofillustration, specific embodiments in which the invention may bepracticed. These embodiments are described in sufficient detail toenable those skilled in the art to implement the present disclosure. Itshould be understood that various embodiments of the present disclosure,although different, are not necessarily mutually exclusive. For example,specific features, structures, and characteristics described herein, inconnection with one embodiment, may be implemented within otherembodiments without departing from the spirit and scope of the presentdisclosure. In addition, it should be understood that the location orarrangement of individual elements within each disclosed embodiment maybe modified without departing from the spirit and scope of the presentdisclosure. The following detailed description is, therefore, not to betaken in a limiting sense, and the scope of the present disclosure isdefined only by the appended claims, appropriately interpreted, alongwith the full range of equivalents to what the claims claim.

Terms used in the specification, ‘first’, ‘second’, etc. can be used todescribe various components, but the components are not to be construedas being limited to the terms. The terms are only used to differentiateone component from other components. For example, the ‘first’ componentmay be named the ‘second’ component without departing from the scope ofthe present invention, and the ‘second’ component may also be similarlynamed the ‘first’ component. The term ‘and/or’ includes a combination ofa plurality of items or any one of a plurality of terms.

It will be understood that when an element is simply referred to asbeing ‘connected to’ or ‘coupled to’ another element without being‘directly connected to’ or ‘directly coupled to’ another element in thepresent description, it may be ‘directly connected to’ or ‘directlycoupled to’ another element or be connected to or coupled to anotherelement, having the other element intervening therebetween. In contrast,it should be understood that when an element is referred to as being“directly coupled” or “directly connected” to another element, there areno intervening elements present.

Furthermore, constitutional parts shown in the embodiments of thepresent invention are independently shown so as to representcharacteristic functions different from each other. Thus, it does notmean that each constitutional part is constituted in a constitutionalunit of separated hardware or software. In other words, eachconstitutional part includes each of enumerated constitutional parts forconvenience. Thus, at least two constitutional parts of eachconstitutional part may be combined to form one constitutional part orone constitutional part may be divided into a plurality ofconstitutional parts to perform each function. The embodiment where eachconstitutional part is combined and the embodiment where oneconstitutional part is divided are also included in the scope of thepresent invention, if not departing from the essence of the presentinvention.

The terms used in the present specification are merely used to describeparticular embodiments, and are not intended to limit the presentinvention. An expression used in the singular encompasses the expressionof the plural, unless it has a clearly different meaning in the context.In the present specification, it is to be understood that terms such as“including”, “having”, etc. are intended to indicate the existence ofthe features, numbers, steps, actions, elements, parts, or combinationsthereof disclosed in the specification, and are not intended to precludethe possibility that one or more other features, numbers, steps,actions, elements, parts, or combinations thereof may exist or may beadded. In other words, when a specific element is referred to as being“included”, elements other than the corresponding element are notexcluded, but additional elements may be included in embodiments of thepresent invention or the scope of the present invention.

In addition, some of constituents may not be indispensable constituentsperforming essential functions of the present invention but be selectiveconstituents improving only performance thereof. The present inventionmay be implemented by including only the indispensable constitutionalparts for implementing the essence of the present invention except theconstituents used in improving performance. The structure including onlythe indispensable constituents except the selective constituents used inimproving only performance is also included in the scope of the presentinvention.

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings. In describingexemplary embodiments of the present invention, well-known functions orconstructions will not be described in detail since they mayunnecessarily obscure the understanding of the present invention. Thesame constituent elements in the drawings are denoted by the samereference numerals, and a repeated description of the same elements willbe omitted.

In addition, hereinafter, an image may mean a picture configuring avideo, or may mean the video itself. For example, “encoding or decodingor both of an image” may mean “encoding or decoding or both of a video”,and may mean “encoding or decoding or both of one image among images ofa video.” Here, a picture and the image may have the same meaning.

Term Description

Encoder: may mean an apparatus performing encoding.

Decoder: may mean an apparatus performing decoding.

Parsing: may mean determination of a value of a syntax element byperforming entropy decoding, or may mean the entropy decoding itself.

Block: may mean a sample of an M×N matrix. Here, M and N are positiveintegers, and the block may mean a sample matrix in a two-dimensionalform.

Sample: is a basic unit of a block, and may indicate a value ranging 0to 2^(Bd-1) depending on the bit depth (B_(d)). The sample may mean apixel in the present invention.

Unit: may mean a unit of encoding and decoding of an image. In encodingand decoding an image, the unit may be an area generated by partitioningone image. In addition, the unit may mean a subdivided unit when oneimage is partitioned into subdivided units during encoding or decoding.In encoding and decoding an image, a predetermined process for each unitmay be performed. One unit may be partitioned into sub units that havesizes smaller than the size of the unit. Depending on functions, theunit may mean a block, a macroblock, a coding tree unit, a coding treeblock, a coding unit, a coding block, a prediction unit, a predictionblock, a transform unit, a transform block, etc. In addition, in orderto distinguish a unit from a block, the unit may include a lumacomponent block, a chroma component block of the luma component block,and a syntax element of each color component block. The unit may havevarious sizes and shapes, and particularly, the shape of the unit may bea two-dimensional geometrical figure such as a rectangular shape, asquare shape, a trapezoid shape, a triangular shape, a pentagonal shape,etc. In addition, unit information may include at least one of a unittype indicating the coding unit, the prediction unit, the transformunit, etc., and a unit size, a unit depth, a sequence of encoding anddecoding of a unit, etc.

Reconstructed Neighbor Unit: may mean a reconstructed unit that ispreviously encoded or decoded, and the reconstructed unit isspatially/temporally adjacent to an encoding/decoding target unit. Here,a reconstructed neighbor unit may mean a reconstructed neighbor block.

Neighbor Block: may mean a block adjacent to an encoding/decoding targetblock. The block adjacent to the encoding/decoding target block may meana block having a boundary being in contact with the encoding/decodingtarget block. The neighbor block may mean a block located at an adjacentvertex of the encoding/decoding target block. The neighbor block maymean a reconstructed neighbor block.

Unit Depth: may mean a partitioned degree of a unit. In a treestructure, a root node may be the highest node, and a leaf node may bethe lowest node.

Symbol: may mean a syntax element of the encoding/decoding target unit,a coding parameter, a value of a transform coefficient, etc.

Parameter Set: may mean header information in a structure of thebitstream. The parameter set may include at least one of a videoparameter set, a sequence parameter set, a picture parameter set, or anadaptation parameter set. In addition, the parameter set may mean sliceheader information and tile header information, etc.

Bitstream: may mean a bit string including encoded image information.

Prediction Unit: may mean a basic unit when performing inter predictionor intra prediction, and compensation for the prediction. One predictionunit may be partitioned into a plurality of partitions. In this case,each of the plurality of partitions may be a basic unit while performingthe predictions and the compensation, and each partition partitionedfrom the prediction unit may be a prediction unit. In addition, oneprediction unit may be partitioned into a plurality of small predictionunits. A prediction unit may have various sizes and shapes, andparticularly, the shape of the prediction unit may be a two-dimensionalgeometrical figure such as a rectangular shape, a square shape, atrapezoid shape, a triangular shape, a pentagonal shape, etc.

Prediction Unit Partition: may mean the shape of a partitionedprediction unit.

Reference Picture List: may mean a list including at least one referencepicture that is used for inter prediction or motion compensation. Typesof the reference picture list may be List Combined (LC), List 0 (L0),List 1 (L1), List 2 (L2), List 3 (L3), etc. At least one referencepicture list may be used for inter prediction.

Inter-Prediction Indicator: may mean one of the inter-predictiondirection (one-way directional prediction, bidirectional prediction,etc.) of an encoding/decoding target block in a case of interprediction, the number of reference pictures used for generating aprediction block by the encoding/decoding target block, and the numberof reference blocks used for performing inter prediction or motioncompensation by the encoding/decoding target block.

Reference Picture Index: may mean an index of a specific referencepicture in the reference picture list.

Reference Picture: may mean a picture to which a specific unit refersfor inter prediction or motion compensation. A reference image may bereferred to as the reference picture.

Motion Vector: is a two-dimensional vector used for inter prediction ormotion compensation, and may mean an offset between an encoding/decodingtarget picture and the reference picture. For example, (mvX, mvY) mayindicate the motion vector, mvX may indicate a horizontal component, andmvY may indicate a vertical component.

Motion Vector Candidate: may mean a unit that becomes a predictioncandidate when predicting the motion vector, or may mean a motion vectorof the unit.

Motion Vector Candidate List: may mean a list configured by using themotion vector candidate.

Motion Vector Candidate Index: may mean an indicator that indicates themotion vector candidate in the motion vector candidate list. The motionvector candidate index may be referred to as an index of a motion vectorpredictor.

Motion Information: may mean the motion vector, the reference pictureindex, and inter-prediction indicator as well as information includingat least one of reference picture list information, the referencepicture, the motion vector candidate, the motion vector candidate index,etc.

Merge Candidate List: may mean a list configured by using the mergecandidate.

Merge Candidate: may include a spatial merge candidate, a temporal mergecandidate, a combined merge candidate, a combined bi-prediction mergecandidate, a zero merge candidate, etc. The merge candidate may includemotion information such as prediction type information, a referencepicture index for each list, a motion vector, etc.

Merge Index: may mean information indicating the merge candidate in themerge candidate list. In addition, the merge index may indicate a block,which derives the merge candidate, among reconstructed blocksspatially/temporally adjacent to the current block. In addition, themerge index may indicate at least one of pieces of motion information ofthe merge candidate.

Transform Unit: may mean a basic unit when performing encoding/decodingof a residual signal, similar to transform, inverse transform,quantization, dequantization, and transform coefficientencoding/decoding. One transform unit may be partitioned into aplurality of small transform units. The transform unit may have varioussizes and shapes. Particularly, the shape of the transform unit may be atwo-dimensional geometrical figure such as a rectangular shape, a squareshape, a trapezoid shape, a triangular shape, a pentagonal shape, etc.

Scaling: may mean a process of multiplying a factor to a transformcoefficient level, and as a result, a transform coefficient may begenerated. The scaling may be also referred to as dequantization.

Quantization Parameter: may mean a value used in scaling the transformcoefficient level during quantization and dequantization. Here, thequantization parameter may be a value mapped to a step size of thequantization.

Delta Quantization Parameter: may mean a difference value between apredicted quantization parameter and a quantization parameter of theencoding/decoding target unit.

Scan: may mean a method of sorting coefficient orders within a block ora matrix. For example, sorting a two-dimensional matrix into aone-dimensional matrix may be referred to as scanning, and sorting aone-dimensional matrix into a two-dimensional matrix may be referred toas scanning or inverse scanning.

Transform Coefficient: may mean a coefficient value generated afterperforming a transform. In the present invention, a quantized transformcoefficient level that is a transform coefficient to which thequantization is applied may be referred to as the transform coefficient.

Non-zero Transform Coefficient: may mean a transform coefficient inwhich a value thereof is not 0, or may mean a transform coefficientlevel in which a value thereof is not 0.

Quantization Matrix: may mean a matrix used in quantization anddequantization in order to enhance subject quality or object quality ofan image. The quantization matrix may be referred to as a scaling list.

Quantization Matrix Coefficient: may mean each element of a quantizationmatrix. The quantization matrix coefficient may be referred to as amatrix coefficient.

Default Matrix: may mean a predetermined quantization matrix that isdefined in the encoder and the decoder in advance.

Non-default Matrix: may mean a quantization matrix that istransmitted/received by a user without being previously defined in theencoder and the decoder.

Coding Tree Unit: may be composed of one luma component (Y) coding treeunit and related two chroma components (Cb, Cr) coding tree units. Eachcoding tree unit may be partitioned by using at least one partitionmethod such as a quad tree, a binary tree, etc. to configure sub unitssuch as coding units, prediction units, transform units, etc. The codingtree unit may be used as a term for indicating a pixel block that is aprocessing unit in decoding/encoding process of an image, like partitionof an input image.

Coding Tree Block: may be used as a term for indicating one of the Ycoding tree unit, the Cb coding tree unit, and the Cr coding tree unit.

FIG. 1 is a block diagram showing configurations of an encodingapparatus according to an embodiment of the present invention.

The encoding apparatus 100 may be a video encoding apparatus or an imageencoding apparatus. A video may include one or more images. The encodingapparatus 100 may encode the one or more images of the video in order oftime.

Referring to FIG. 1 , the encoding apparatus 100 may include a motionprediction unit 111, a motion compensation unit 112, an intra-predictionunit 120, a switch 115, a subtractor 125, a transform unit 130, aquantization unit 140, an entropy encoding unit 150, a dequantizationunit 160, an inverse transform unit 170, an adder 175, a filter unit180, and a reference picture buffer 190.

The encoding apparatus 100 may encode an input picture in an intra modeor an inter mode or both. In addition, the encoding apparatus 100 maygenerate a bitstream by encoding the input picture, and may output thegenerated bitstream. When the intra mode is used as a prediction mode,the switch 115 may be switched to intra. When the inter mode is used asa prediction mode, the switch 115 may be switched to inter. Here, theintra mode may be referred to as an intra-prediction mode, and the intermode may be referred to as an inter-prediction mode. The encodingapparatus 100 may generate a prediction block of an input block of theinput picture. In addition, after generating the prediction block, theencoding apparatus 100 may encode residuals between the input block andthe prediction block. The input picture may be referred to as a currentimage that is a target of current encoding. The input block may bereferred to as a current block or as an encoding target block that is atarget of the current encoding.

When the prediction mode is the intra mode, the intra-prediction unit120 may use a pixel value of a previously encoded block, which isadjacent to the current block, as a reference pixel. Theintra-prediction unit 120 may perform spatial prediction by using thereference pixel, and may generate prediction samples of the input blockby using the spatial prediction. Here, intra prediction may meanintra-frame prediction.

When the prediction mode is the inter mode, the motion prediction unit111 may search for a region that is optimally matched with the inputblock from a reference picture in a motion predicting process, and mayderive a motion vector by using the searched region. The referencepicture may be stored in the reference picture buffer 190.

The motion compensation unit 112 may generate the prediction block byperforming motion compensation using the motion vector. Here, the motionvector may be a two-dimensional vector that is used for interprediction. In addition, the motion vector may indicate offset betweenthe current picture and the reference picture. Here, inter predictionmay be mean inter-frame prediction.

When a value of the motion vector is not an integer, the motionprediction unit 111 and the motion compensation unit 112 may generatethe prediction block by applying an interpolation filter to a partialregion in the reference picture. In order to perform inter prediction ormotion compensation, on the basis of the coding unit, it is possible todetermine which methods the motion prediction and compensation methodsof a prediction unit in the coding unit uses among the skip mode, themerge mode, the AMVP mode, and a current picture reference mode. Interprediction or motion compensation may be performed according to eachmode. Here, the current picture reference mode may mean a predictionmode using a pre-reconstructed region of a current picture having anencoding target block. In order to specify the pre-reconstructed region,a motion vector for the current picture reference mode may be defined.Whether the encoding target block is encoded in the current picturereference mode may be encoded by using a reference picture index of theencoding target block.

The subtractor 125 may generate a residual block by using a differencebetween the input block and the prediction block. The residual block maybe referred to as a residual signal.

The transform unit 130 may generate a transform coefficient bytransforming the residual block, and may output the transformcoefficient. Here, the transform coefficient may be a coefficient valuegenerated by transforming the residual block. In a transform skip mode,the transform unit 130 may skip the transforming of the residual block.

A quantized transform coefficient level may be generated by applyingquantization to the transform coefficient. Hereinafter, the quantizedtransform coefficient level may be referred to as the transformcoefficient in the embodiment of the present invention.

The quantization unit 140 may generate the quantized transformcoefficient level by quantizing the transform coefficient depending onthe quantization parameter, and may output the quantized transformcoefficient level. Here, the quantization unit 140 may quantize thetransform coefficient by using a quantization matrix.

The entropy encoding unit 150 may generate the bitstream by performingentropy encoding according to the probability distribution, on valuescalculated by the quantization unit 140 or on coding parameter valuescalculated in an encoding process, etc., and may output the generatedbitstream. The entropy encoding unit 150 may perform the entropyencoding on information for decoding an image, and on information of apixel of an image. For example, the information for decoding an imagemay include a syntax element, etc.

When the entropy encoding is applied, symbols are represented byallocating a small number of bits to the symbols having high occurrenceprobability and allocating a large number of bits to the symbols havinglow occurrence probability, thereby reducing the size of the bitstreamof encoding target symbols. Therefore, compression performance of theimage encoding may be increased through the entropy encoding. For theentropy encoding, the entropy encoding unit 150 may use an encodingmethod such as exponential Golomb, context-adaptive variable lengthcoding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).For example, the entropy encoding unit 150 may perform the entropyencoding by using a variable length coding/code (VLC) table. Inaddition, the entropy encoding unit 150 may derive a binarization methodof the target symbol and a probability model of the target symbol/bin,and may perform arithmetic coding by using the derived binarizationmethod or the derived probability model thereafter.

In order to encode the transform coefficient level, the entropy encodingunit 150 may change a two-dimensional block form coefficient into aone-dimensional vector form by using a transform coefficient scanningmethod. For example, the two-dimensional form coefficient may be changedinto the one-dimensional vector form by scanning the coefficient of theblock with up-right scanning. According to the size of the transformunit and the intra-prediction mode, instead of the up-right scanning, itis possible to use vertical direction scanning for scanning thetwo-dimensional block form coefficient in a column direction, andhorizontal direction scanning for scanning the two-dimensional blockform coefficient in a row direction. That is, it is possible todetermine which scanning method among up-right scanning, verticaldirection scanning, and horizontal direction scanning is to be useddepending on the size of the transform unit and the intra-predictionmode.

The coding parameter may include information, such as the syntaxelement, which is encoded by the encoder and is transmitted to thedecoder, and may include information that may be derived in the encodingor decoding process. The coding parameter may mean information that isnecessary to encode or decode an image. For example, the codingparameter may include at least one value or combined form of the blocksize, the block depth, the block partition information, the unit size,the unit depth, the unit partition information, the partition flag of aquad-tree form, the partition flag of a binary-tree form, the partitiondirection of a binary-tree form, the intra-prediction mode, theintra-prediction direction, the reference sample filtering method, theprediction block boundary filtering method, the filter tap, the filtercoefficient, the inter-prediction mode, the motion information, themotion vector, the reference picture index, the inter-predictiondirection, the inter-prediction indicator, the reference picture list,the motion vector predictor, the motion vector candidate list, theinformation about whether or not the motion merge mode is used, themotion merge candidate, motion merge candidate list, the informationabout whether or not the skip mode is used, interpolation filter type,the motion vector size, accuracy of motion vector representation, thetransform type, the transform size, the information about whetheradditional (secondary) transform is used, the information about whetheror not a residual signal is present, the coded block pattern, the codedblock flag, the quantization parameter, the quantization matrix, thefilter information within a loop, the information about whether or not afilter is applied within a loop, the filter coefficient within a loop,binarization/inverse binarization method, the context model, the contextbin, the bypass bin, the transform coefficient, transform coefficientlevel, transform coefficient level scanning method, the imagedisplay/output order, slice identification information, slice type,slice partition information, tile identification information, tile type,tile partition information, the picture type, bit depth, and theinformation of a luma signal or a chroma signal.

The residual signal may mean the difference between the original signaland the prediction signal. Alternatively, the residual signal may be asignal generated by transforming the difference between the originalsignal and the prediction signal. Alternatively, the residual signal maybe a signal generated by transforming and quantizing the differencebetween the original signal and the prediction signal. The residualblock may be the residual signal of a block unit.

When the encoding apparatus 100 performs encoding by using interprediction, the encoded current picture may be used as a referencepicture for another image(s) that will be processed thereafter.Accordingly, the encoding apparatus 100 may decode the encoded currentpicture, and may store the decoded image as the reference picture. Inorder to perform the decoding, dequantization and inverse transform maybe performed on the encoded current picture.

A quantized coefficient may be dequantized by the dequantization unit160, and may be inversely transformed by the inverse transform unit 170.The dequantized and inversely transformed coefficient may be added tothe prediction block by the adder 175, whereby a reconstructed block maybe generated.

The reconstructed block may pass the filter unit 180. The filter unit180 may apply at least one of a deblocking filter, a sample adaptiveoffset (SAO), and an adaptive loop filter (ALF) to the reconstructedblock or a reconstructed picture. The filter unit 180 may be referred toas an in-loop filter.

The deblocking filter may remove block distortion that occurs atboundaries between the blocks. In order to determine whether or not thedeblocking filter is operated, it is possible to determine whether ornot the deblocking filter is applied to the current block on the basisof the pixels included in several rows or columns in the block. When thedeblocking filter is applied to the block, a strong filter or a weakfilter may be applied depending on required deblocking filteringstrength. In addition, in applying the deblocking filter, horizontaldirection filtering and vertical direction filtering may be processed inparallel.

The sample adaptive offset may add an optimum offset value to the pixelvalue in order to compensate for an encoding error. The sample adaptiveoffset may correct an offset between the deblocking filtered image andthe original picture for each pixel. In order to perform the offsetcorrection on a specific picture, it is possible to use a method ofapplying an offset in consideration of edge information of each pixel ora method of partitioning pixels of an image into the predeterminednumber of regions, determining a region to be subjected to perform anoffset correction, and applying the offset correction to the determinedregion.

The adaptive loop filter may perform filtering on the basis of a valueobtained by comparing the reconstructed picture and the originalpicture. Pixels of an image may be partitioned into predeterminedgroups, one filter being applied to each of the groups is determined,and different filtering may be performed at each of the groups.Information about whether or not the adaptive loop filter is applied tothe luma signal may be transmitted for each coding unit (CU). A shapeand a filter coefficient of an adaptive loop filter being applied toeach block may vary. In addition, an adaptive loop filter having thesame form (fixed form) may be applied regardless of characteristics of atarget block.

The reconstructed block that passed the filter unit 180 may be stored inthe reference picture buffer 190.

FIG. 2 is a block diagram showing configurations of a decoding apparatusaccording to an embodiment of the present invention.

The decoding apparatus 200 may be a video decoding apparatus or an imagedecoding apparatus.

Referring to FIG. 2 , the decoding apparatus 200 may include an entropydecoding unit 210, a dequantization unit 220, an inverse transform unit230, an intra-prediction unit 240, a motion compensation unit 250, anadder 255, a filter unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bitstream outputted from theencoding apparatus 100. The decoding apparatus 200 may decode thebitstream in the intra mode or the inter mode. In addition, the decodingapparatus 200 may generate a reconstructed picture by performingdecoding, and may output the reconstructed picture.

When a prediction mode used in decoding is the intra mode, the switchmay be switched to intra. When the prediction mode used in decoding isthe inter mode, the switch may be switched to inter.

The decoding apparatus 200 may obtain the reconstructed residual blockfrom the inputted bitstream, and may generate the prediction block. Whenthe reconstructed residual block and the prediction block are obtained,the decoding apparatus 200 may generate the reconstructed block, whichis a decoding target block, by adding the reconstructed residual blockand the prediction block. The decoding target block may be referred toas a current block.

The entropy decoding unit 210 may generate symbols by performing entropydecoding on the bitstream according to the probability distribution. Thegenerated symbols may include a symbol having a quantized transformcoefficient level. Here, a method of entropy decoding may be similar tothe above-described method of the entropy encoding. For example, themethod of the entropy decoding may be an inverse process of theabove-described method of the entropy encoding.

In order to decode the transform coefficient level, the entropy decodingunit 210 may perform transform coefficient scanning, whereby theone-dimensional vector form coefficient can be changed into thetwo-dimensional block form. For example, the one-dimensional vector formcoefficient may be changed into a two-dimensional block form by scanningthe coefficient of the block with up-right scanning. According to thesize of the transform unit and the intra-prediction mode, instead ofup-right scanning, it is possible to use vertical direction scanning andhorizontal direction scanning. That is, it is possible to determinewhich scanning method among up-right scanning, vertical directionscanning, and horizontal direction scanning is used depending on thesize of the transform unit and the intra-prediction mode.

The quantized transform coefficient level may be dequantized by thedequantization unit 220, and may be inversely transformed by the inversetransform unit 230. The quantized transform coefficient level isdequantized and is inversely transformed so as to generate areconstructed residual block. Here, the dequantization unit 220 mayapply the quantization matrix to the quantized transform coefficientlevel.

When the intra mode is used, the intra-prediction unit 240 may generatea prediction block by performing the spatial prediction that uses thepixel value of the previously decoded block that is adjacent to thedecoding target block.

When the inter mode is used, the motion compensation unit 250 maygenerate the prediction block by performing motion compensation thatuses both the motion vector and the reference picture stored in thereference picture buffer 270. When the value of the motion vector is notan integer, the motion compensation unit 250 may generate the predictionblock by applying the interpolation filter to the partial region in thereference picture. In order to perform motion compensation, on the basisof the coding unit, it is possible to determine which method the motioncompensation method of a prediction unit in the coding unit uses amongthe skip mode, the merge mode, the AMVP mode, and a current picturereference mode. In addition, it is possible to perform motioncompensation depending on the modes. Here, the current picture referencemode may mean a prediction mode using a previously reconstructed regionwithin the current picture having the decoding target block. Thepreviously reconstructed region may not be adjacent to the decodingtarget block. In order to specify the previously reconstructed region, afixed vector may be used for the current picture reference mode. Inaddition, a flag or an index indicating whether or not the decodingtarget block is a block decoded in the current picture reference modemay be signaled, and may be derived by using the reference picture indexof the decoding target block. The current picture for the currentpicture reference mode may exist at a fixed position (for example, aposition of a reference picture index is 0 or the last position) withinthe reference picture list for the decoding target block. In addition,it is possible for the current picture to be variably positioned withinthe reference picture list, and to this end, it is possible to signalthe reference picture index indicating a position of the currentpicture. Here, signaling a flag or an index may mean that the encoderentropy encodes the corresponding flag or index and includes into abitstream, and that the decoder entropy decodes the corresponding flagor index from the bitstream.

The reconstructed residual block may be added to the prediction block bythe adder 255. A block generated by adding the reconstructed residualblock and the prediction block may pass the filter unit 260. The filterunit 260 may apply at least one of the deblocking filter, the sampleadaptive offset, and the adaptive loop filter to the reconstructed blockor to the reconstructed picture. The filter unit 260 may output thereconstructed picture. The reconstructed picture may be stored in thereference picture buffer 270, and may be used for inter prediction.

FIG. 3 is a view schematically showing a partition structure of an imagewhen encoding and decoding the image. FIG. 3 schematically shows anembodiment of partitioning one unit into a plurality of sub-units.

In order to efficiently partition an image, a coding unit (CU) may beused in encoding and decoding. Here, the coding unit may mean anencoding unit. The unit may be a combination of 1) a syntax element and2) a block including image samples. For example, “partition of a unit”may mean “partition of a block relative to a unit”. The block partitioninformation may include information about the unit depth. Depthinformation may indicate the number of times a unit is partitioned or apartitioned degree of a unit or both.

Referring to FIG. 3 , an image 300 is sequentially partitioned for eachlargest coding unit (LCU), and a partition structure is determined foreach LCU. Here, the LCU and a coding tree unit (CTU) have the samemeaning. One unit may have depth information based on a tree structure,and may be hierarchically partitioned. Each of the partitioned sub-unitsmay have depth information. The depth information indicates the numberof times a unit is partitioned or a partitioned degree of a unit orboth, and thus, the depth information may include information about thesize of the sub-unit.

The partition structure may mean distribution of a coding unit (CU) inthe LCU 310. The CU may be a unit for efficiently encoding/decoding animage. The distribution may be determined on the basis of whether or notone CU will be partitioned in plural (a positive integer equal to ormore than 2 including 2, 4, 8, 16, etc.). The width size and the heightsize of the CUs resulting from the partitioning may respectively be ahalf width size and a half height size of the original CU.Alternatively, according to the number of partitionings, the width sizeand the height size of the partitioned CU may respectively be smallerthan the width size and the height size of the original CU. Thepartitioned CU may be recursively partitioned into a plurality offurther partitioned CUs, wherein the further partitioned CU has a widthsize and a height size smaller than those of the partitioned CU in thesame partition method.

Here, the partition of a CU may be recursively performed up to apredetermined depth. Depth information may be information indicating asize of the CU, and may be stored for each CU. For example, the depth ofthe LCU may be 0, and the depth of a smallest coding unit (SCU) may be apredetermined maximum depth. Here, the LCU may be a coding unit having amaximum size as described above, and the SCU may be a coding unit havinga minimum size.

Whenever the LCU 310 begins to be partitioned, and the width size andthe height size of the CU are decreased by the partitioning, the depthof a CU is increased by 1. In a case of a CU which cannot bepartitioned, the CU may have a 2N×2N size for each depth. In a case of aCU that can be partitioned, the CU having a 2N×2N size may bepartitioned into a plurality of N×N-size CUs. The size of N is reducedby half whenever the depth is increased by 1.

For example, when one coding unit is partitioned into four sub-codingunits, a width size and a height size of one of the four sub-codingunits may respectively be a half width size and a half height size ofthe original coding unit. For example, when a 32×32-size coding unit ispartitioned into four sub-coding units, each of the four sub-codingunits may have a 16×16 size. When one coding unit is partitioned intofour sub-coding units, the coding unit may be partitioned in a quad-treeform.

For example, when one coding unit is partitioned into two sub-codingunits, a width size or a height size of one of the two sub-coding unitsmay respectively be a half width size or a half height size of theoriginal coding unit. For example, when a 32×32-size coding unit isvertically partitioned into two sub-coding units, each of the twosub-coding units may have a 16×32 size. For example, when a 32×32-sizecoding unit is horizontally partitioned into two sub-coding units, eachof the two sub-coding units may have a 32×16 size. When one coding unitis partitioned into two sub-coding units, the coding unit may bepartitioned in a binary-tree form.

Referring to FIG. 3 , the LCU having a minimum depth of 0 may be 64×64pixels, and the SCU having a maximum depth of 3 may be 8×8 pixels. Here,a CU having 64×64 pixels, which is the LCU, may be denoted by a depth of0, a CU having 32×32 pixels may be denoted by a depth of 1, a CU having16×16 pixels may be denoted by a depth of 2, and a CU having 8×8 pixels,which is the SCU, may be denoted by a depth of 3.

In addition, information about whether or not a CU will be partitionedmay be represented through partition information of a CU. The partitioninformation may be 1 bit information. The partition information may beincluded in all CUs other than the SCU. For example, when a value of thepartition information is 0, a CU may not be partitioned, and when avalue of the partition information is 1, a CU may be partitioned.

FIG. 4 is a view showing forms of a prediction unit (PU) that may beincluded in a coding unit (CU).

A CU that is no longer partitioned, from among CUs partitioned from theLCU, may be partitioned into at least one prediction unit (PU). Thisprocess may be also referred to as a partition.

The PU may be a basic unit for prediction. The PU may be encoded anddecoded in any one of a skip mode, an inter mode, and an intra mode. ThePU may be partitioned in various forms depending on the modes.

In addition, the coding unit may not be partitioned into a plurality ofprediction units, and the coding unit and the prediction unit have thesame size.

As shown in FIG. 4 , in the skip mode, the CU may not be partitioned. Inthe skip mode, a 2N×2N mode 410 having the same size as a CU withoutpartition may be supported.

In the inter mode, 8 partitioned forms may be supported within a CU. Forexample, in the inter mode, the 2N×2N mode 410, a 2N×N mode 415, an N×2Nmode 420, an N×N mode 425, a 2N×nU mode 430, a 2N×nD mode 435, an nL×2Nmode 440, and an nR×2N mode 445 may be supported. In the intra mode, the2N×2N mode 410 and the N×N mode 425 may be supported.

One coding unit may be partitioned into one or more prediction units.One prediction unit may be partitioned into one or more sub-predictionunits.

For example, when one prediction unit is partitioned into foursub-prediction units, a width size and a height size of one of the foursub-prediction units may be a half width size and a half height size ofthe original prediction unit. For example, when a 32×32-size predictionunit is partitioned into four sub-prediction units, each of the foursub-prediction units may have a 16×16 size. When one prediction unit ispartitioned into four sub-prediction units, the prediction unit may bepartitioned in the quad-tree form.

For example, when one prediction unit is partitioned into twosub-prediction units, a width size or a height size of one of the twosub-prediction units may be a half width size or a half height size ofthe original prediction unit. For example, when a 32×32-size predictionunit is vertically partitioned into two sub-prediction units, each ofthe two sub-prediction units may have a 16×32 size. For example, when a32×32-size prediction unit is horizontally partitioned into twosub-prediction units, each of the two sub-prediction units may have a32×16 size. When one prediction unit is partitioned into twosub-prediction units, the prediction unit may be partitioned in thebinary-tree form.

FIG. 5 is a view showing forms of a transform unit (TU) that may beincluded in a coding unit (CU).

A transform unit (TU) may be a basic unit used for transform,quantization, inverse transform, and dequantization within a CU. The TUmay have a square shape or a rectangular shape, etc. The TU may bedependently determined by a size of a CU or a form of a CU or both.

A CU that is no longer partitioned among CUs partitioned from the LCUmay be partitioned into at least one TU. Here, the partition structureof the TU may be a quad-tree structure. For example, as shown in FIG. 5, one CU 510 may be partitioned once or more depending on the quad-treestructure. The case where one CU is partitioned at least once may bereferred to as recursive partition. Through the partitioning, one CU 510may be formed of TUs having various sizes. Alternatively, a CU may bepartitioned into at least one TU depending on the number of verticallines partitioning the CU or the number of horizontal lines partitioningthe CU or both. The CU may be partitioned into TUs that are symmetricalto each other, or may be partitioned into TUs that are asymmetrical toeach other. In order to partition the CU into TUs that are symmetricalto each other, information of a size/shape of the TU may be signaled,and may be derived from information of a size/shape of the CU.

In addition, the coding unit may not be partitioned into transformunits, and the coding unit and the transform unit may have the samesize.

One coding unit may be partitioned into at least one transform unit, andone transform unit may be partitioned into at least one sub-transformunit.

For example, when one transform unit is partitioned into foursub-transform units, a width size and a height size of one of the foursub-transform units may respectively be a half width size and a halfheight size of the original transform unit. For example, when a32×32-size transform unit is partitioned into four sub-transform units,each of the four sub-transform units may have a 16×16 size. When onetransform unit is partitioned into four sub-transform units, thetransform unit may be partitioned in the quad-tree form.

For example, when one transform unit is partitioned into twosub-transform units, a width size or a height size of one of the twosub-transform units may respectively be a half width size or a halfheight size of the original transform unit. For example, when a32×32-size transform unit is vertically partitioned into twosub-transform units, each of the two sub-transform units may have a16×32 size. For example, when a 32×32-size transform unit ishorizontally partitioned into two sub-transform units, each of the twosub-transform units may have a 32×16 size. When one transform unit ispartitioned into two sub-transform units, the transform unit may bepartitioned in the binary-tree form.

When performing transform, the residual block may be transformed byusing at least one of predetermined transform methods. For example, thepredetermined transform methods may include discrete cosine transform(DCT), discrete sine transform (DST), KLT, etc. Which transform methodis applied to transform the residual block may be determined by using atleast one of inter-prediction mode information of the prediction unit,intra-prediction mode information of the prediction unit, and size/shapeof the transform block. Information indicating the transform method maybe signaled.

FIG. 6 is a view for explaining an embodiment of a process of intraprediction.

The intra-prediction mode may be a non-directional mode or a directionalmode. The non-directional mode may be a DC mode or a planar mode. Thedirectional mode may be a prediction mode having a particular directionor angle, and the number of directional modes may be M which is equal toor greater than one. The directional mode may be indicated as at leastone of a mode number, a mode value, and a mode angle.

The number of intra-prediction modes may be N which is equal to orgreater than one, including the non-directional and directional modes.

The number of intra-prediction modes may vary depending on the size of ablock. For example, when the size is 4×4 or 8×8, the number may be 67,and when the size is 16×16, the number may be 35, and when the size is32×32, the number may be 19, and when the size is 64×64, the number maybe 7.

The number of intra-prediction modes may be fixed to N regardless of thesize of a block. For example, the number may be fixed to at least one of35 or 67 regardless of the size of a block.

The number of intra-prediction modes may vary depending on a type of acolor component. For example, the number of prediction modes may varydepending on whether a color component is a luma signal or a chromasignal.

Intra encoding and/or decoding may be performed by using a sample valueor an encoding parameter included in a reconstructed neighboring block.

For encoding/decoding a current block in intra prediction, whether ornot samples included in a reconstructed neighboring block are availableas reference samples of an encoding/decoding target block may beidentified. When there are samples that cannot be used as referencesamples of the encoding/decoding target block, sample values are copiedand/or interpolated into the samples that cannot be used as thereference samples by using at least one of samples included in thereconstructed neighboring block, whereby the samples that cannot be usedas reference samples can be used as the reference samples of theencoding/decoding target block.

In intra prediction, based on at least one of an intra-prediction modeand the size of the encoding/decoding target block, a filter may beapplied to at least one of a reference sample or a prediction sample.Here, the encoding/decoding target block may mean a current block, andmay mean at least one of a coding block, a prediction block, and atransform block. A type of a filter being applied to a reference sampleor a prediction sample may vary depending on at least one of theintra-prediction mode or size/shape of the current block. The type ofthe filter may vary depending on at least one of the number of filtertaps, a filter coefficient value, or filter strength.

In a non-directional planar mode among intra-prediction modes, whengenerating a prediction block of the encoding/decoding target block, asample value in the prediction block may be generated by using aweighted sum of an upper reference sample of the current sample, a leftreference sample of the current sample, an upper right reference sampleof the current block, and a lower left reference sample of the currentblock according to the sample location.

In a non-directional DC mode among intra-prediction modes, whengenerating a prediction block of the encoding/decoding target block, itmay be generated by an average value of upper reference samples of thecurrent block and left reference samples of the current block. Inaddition, filtering may be performed on one or more upper rows and oneor more left columns adjacent to the reference sample in theencoding/decoding block by using reference sample values.

In a case of multiple directional modes (angular mode) amongintra-prediction modes, a prediction block may be generated by using theupper right and/or lower left reference sample, and the directionalmodes may have different direction. In order to generate a predictionsample value, interpolation of a real number unit may be performed.

In order to perform an intra-prediction method, an intra-prediction modeof a current prediction block may be predicted from an intra-predictionmode of a neighboring prediction block that is adjacent to the currentprediction block. In a case of prediction the intra-prediction mode ofthe current prediction block by using mode information predicted fromthe neighboring intra-prediction mode, when the current prediction blockand the neighboring prediction block have the same intra-predictionmode, information that the current prediction block and the neighboringprediction block have the same intra-prediction mode may be transmittedby using predetermined flag information. When the intra-prediction modeof the current prediction block is different from the intra-predictionmode of the neighboring prediction block, intra-prediction modeinformation of the encoding/decoding target block may be encoded byperforming entropy encoding.

FIG. 7 is a view for explaining an embodiment of a process of interprediction.

The quadrangular shapes shown in FIG. 7 may indicate images (or,pictures). Also, the arrows of FIG. 7 may indicate predictiondirections. That is, images may be encoded or decoded or both accordingto prediction directions. Each image may be classified into an I-picture(intra picture), a P-picture (uni-predictive picture), a B-picture(bi-predictive picture), etc. according to encoding types. Each picturemay be encoded and decoded depending on an encoding type of eachpicture.

When an image, which is an encoding target, is an I-picture, the imageitself may be intra encoded without inter prediction. When an image,which is an encoding target, is a P-picture, the image may be encoded byinter prediction or motion compensation using a reference picture onlyin a forward direction. When an image, which is an encoding target, is aB-picture, the image may be encoded by inter prediction or motioncompensation using reference pictures in both a forward direction and areverse direction. Alternatively, the image may be encoded by interprediction or motion compensation using a reference picture in one of aforward direction and a reverse direction. Here, when aninter-prediction mode is used, the encoder may perform inter predictionor motion compensation, and the decoder may perform motion compensationin response to the encoder. Images of the P-picture and the B-picturethat are encoded or decoded or both by using a reference picture may beregarded as an image for inter prediction.

Hereinafter, inter prediction according to an embodiment will bedescribed in detail.

Inter prediction or motion compensation may be performed by using both areference picture and motion information. In addition, inter predictionmay use the above described skip mode.

The reference picture may be at least one of a previous picture and asubsequent picture of a current picture. Here, inter prediction maypredict a block of the current picture depending on the referencepicture. Here, the reference picture may mean an image used inpredicting a block. Here, an area within the reference picture may bespecified by using a reference picture index (refIdx) indicating areference picture, a motion vector, etc.

Inter prediction may select a reference picture and a reference blockrelative to a current block within the reference picture. A predictionblock of the current block may be generated by using the selectedreference block. The current block may be a block that is a currentencoding or decoding target among blocks of the current picture.

Motion information may be derived from a process of inter prediction bythe encoding apparatus 100 and the decoding apparatus 200. In addition,the derived motion information may be used in performing interprediction. Here, the encoding apparatus 100 and the decoding apparatus200 may enhance encoding efficiency or decoding efficiency or both byusing motion information of a reconstructed neighboring block or motioninformation of a collocated block (col block) or both. The col block maybe a block relative to a spatial position of the encoding/decodingtarget block within a collocated picture (col picture) that ispreviously reconstructed. The reconstructed neighboring block may be ablock within a current picture, and a block that is previouslyreconstructed through encoding or decoding or both. In addition, thereconstructed block may be a block adjacent to the encoding/decodingtarget block or a block positioned at an outer corner of theencoding/decoding target block or both. Here, the block positioned atthe outer corner of the encoding/decoding target block may be a blockthat is vertically adjacent to a neighboring block horizontally adjacentto the encoding/decoding target block. Alternatively, the blockpositioned at the outer corner of the encoding/decoding target block maybe a block that is horizontally adjacent to a neighboring blockvertically adjacent to the encoding/decoding target block.

The encoding apparatus 100 and the decoding apparatus 200 mayrespectively determine a block that exists at a position spatiallyrelative to the encoding/decoding target block within the col picture,and may determine a predefined relative position on the basis of thedetermined block. The predefined relative position may be an innerposition or an outer position or both of a block that exists at aposition spatially relative to the encoding/decoding target block. Inaddition, the encoding apparatus 100 and the decoding apparatus 200 mayrespectively derive the col block on the basis of the determinedpredefined relative position. Here, the col picture may be one pictureof at least one reference picture included in the reference picturelist.

A method of deriving the motion information may vary according to aprediction mode of the encoding/decoding target block. For example, aprediction mode being applied for inter prediction may include anadvanced motion vector prediction (AMVP), a merge mode, etc. Here, themerge mode may be referred to as a motion merge mode.

For example, when AMVP is applied as the prediction mode, the encodingapparatus 100 and the decoding apparatus 200 may respectively generate amotion vector candidate list by using a motion vector of thereconstructed neighboring block or a motion vector of the col block orboth. The motion vector of the reconstructed neighboring block or themotion vector of the col block or both may be used as motion vectorcandidates. Here, the motion vector of the col block may be referred toas a temporal motion vector candidate, and the motion vector of thereconstructed neighboring block may be referred to as a spatial motionvector candidate.

The encoding apparatus 100 may generate a bitstream, and the bitstreammay include a motion vector candidate index. That is, the encodingapparatus 100 may generate a bitstream by entropy encoding the motionvector candidate index. The motion vector candidate index may indicatean optimum motion vector candidate that is selected from motion vectorcandidates included in the motion vector candidate list. The motionvector candidate index may be transmitted from the encoding apparatus100 to the decoding apparatus 200 through the bitstream.

The decoding apparatus 200 may entropy decode the motion vectorcandidate index from the bitstream, and may select a motion vectorcandidate of a decoding target block among the motion vector candidatesincluded in the motion vector candidate list by using the entropydecoded motion vector candidate index.

The encoding apparatus 100 may calculate a motion vector difference(MVD) between the motion vector and the motion vector candidate of thedecoding target block, and may entropy encode the MVD. The bitstream mayinclude the entropy encoded MVD. The MVD may be transmitted from theencoding apparatus 100 to the decoding apparatus 200 through thebitstream. Here, the decoding apparatus 200 may entropy decode thereceived MVD from the bitstream. The decoding apparatus 200 may derive amotion vector of the decoding target block through a sum of the decodedMVD and the motion vector candidate.

The bitstream may include a reference picture index indicating areference picture, etc., and a reference picture index may be entropyencoded and transmitted from the encoding apparatus 100 to the decodingapparatus 200 through the bitstream. The decoding apparatus 200 maypredict a motion vector of the decoding target block by using motioninformation of neighboring blocks, and may derive the motion vector ofthe decoding target block by using the predicted motion vector and themotion vector difference. The decoding apparatus 200 may generate theprediction block of the decoding target block on the basis of thederived motion vector and reference picture index information.

As another method of deriving the motion information, a merge mode isused. The merge mode may mean a merger of motions of a plurality ofblocks. The merge mode may mean application of motion information of oneblock to another block. When the merge mode is applied, the encodingapparatus 100 and the decoding apparatus 200 may respectively generate amerge candidate list by using motion information of the reconstructedneighboring block or motion information of the col block or both. Themotion information may include at least one of 1) the motion vector, 2)the reference picture index, and 3) the inter-prediction indicator. Aprediction indicator may indicate a uni-direction (L0 prediction, L1prediction) or a bi-direction.

Here, the merge mode may be applied to each CU or each PU. When themerge mode is performed at each CU or each PU, the encoding apparatus100 may generate a bitstream by entropy decoding predefined information,and may transmit the bitstream to the decoding apparatus 200. Thebitstream may include the predefined information. The predefinedinformation may include: 1) a merge flag that is information indicatingwhether or not the merge mode is performed for each block partition; and2) a merge index that is information to which a block among theneighboring blocks adjacent to the encoding target block is merged. Forexample, neighboring blocks adjacent to the encoding target block mayinclude a left neighboring block of the encoding target block, an upperneighboring block of the encoding target block, a temporally neighboringblock of the encoding target block, etc.

The merge candidate list may indicate a list storing motion information.In addition, the merge candidate list may be generated in advance ofperforming the merge mode. The motion information stored in the mergecandidate list may be at least one of motion information of theneighboring block adjacent to the encoding/decoding target block, motioninformation of the collocated block relative to the encoding/decodingtarget block in the reference picture, motion information newlygenerated by a combination of motion information that exists in themerge motion candidate list in advance, and a zero merge candidate.Here, motion information of the neighboring block adjacent to theencoding/decoding target block may be referred to as a spatial mergecandidate. Motion information of the collocated block relative to theencoding/decoding target block in the reference picture may be referredto as a temporal merge candidate.

A skip mode may be a mode applying the mode information of theneighboring block itself to the encoding/decoding target block. The skipmode may be one of modes used for inter prediction. When the skip modeis used, the encoding apparatus 100 may entropy encode information aboutmotion information of which block is used as motion information of theencoding target block, and may transmit the information to the decodingapparatus 200 through a bitstream. The encoding apparatus 100 may nottransmit other information, for example, syntax element information, tothe decoding apparatus 200. The syntax element information may includeat least one of motion vector difference information, a coded blockflag, and a transform coefficient level.

A residual signal generated after intra or inter prediction may betransformed into a frequency domain through a transform process as apart of a quantization process. Here, a primary transform may use DCTtype 2 (DCT-II) as well as various DCT, DST kernels. On a residualsignal, these transform kernels may perform a separable transformperforming a 1D transform in a horizontal and/or vertical direction, ormay perform a 2D non-separable transform.

For example, DCT and DST types used in transform may use DCT-II, DCT-V,DCT-VIII, DST-I, and DST-VII as shown in following tables in a case ofthe 1D transform. For example, as shown in the table 1 and table 2, aDCT or DST type used in transform by composing a transform set may bederived.

TABLE 1 Transform set Transform 0 DST_VII, DCT-VIH 1 DST-VII, DST-I 2DST-VII, DCT-V

TABLE 2 Transform set Transform 0 DST_VII, DCT-VIII, DST-I 1 DST-VII,DST-I, DCT-VIII 2 DST-VII, DCT-V, DST-I

For example, as shown in FIG. 8 , according to an intra-prediction mode,different transform sets are defined for horizontal and verticaldirections. Next, the encoder/decoder may perform transform and/orinverse transform by using an intra-prediction mode of a currentencoding/decoding target block and transform of a relevant transformset. In this case, entropy encoding/decoding is not performed on thetransform set, and the encoder/decoder may define the transform setaccording to the same rule. In this case, information indicating whichtransform is used among transforms of the transform set may be entropyencoding/decoding. For example, when the size of a block is equal to orless than 64×64, three transform sets are composed as shown in table 2according to an intra-prediction mode, and three transforms are used foreach horizontal direction transform and vertical direction transform tocombine and perform total nine multi-transform methods. Next, a residualsignal is encoded/decoded by using the optimum transform method, wherebyencoding efficiency can be enhanced. Here, in order to perform entropyencoding/decoding on information about which transform method is usedamong three transforms of one transform set, truncated unarybinarization may be used. Here, for at least one of vertical transformand horizontal transform, entropy encoding/decoding may be performed onthe information indicating which transform is used among transforms of atransform set.

After completing the above-described primary transform, the encoder mayperform a secondary transform to increase energy concentration fortransformed coefficients as shown in FIG. 9 . The secondary transformmay perform a separable transform performing a 1D transform in ahorizontal and/or vertical direction, or may perform a 2D non-separabletransform. Used transform information may be transmitted or may bederived by the encoder/decoder according to current and neighboringencoding information. For example, like the 1D transform, a transformset for the secondary transform may be defined. Entropyencoding/decoding is not performed on the transform set, and theencoder/decoder may define the transform set according to the same rule.In this case, information indicating which transform is used amongtransforms of the transform set may be transmitted, and the informationmay be applied to at least one residual signal through intra or interprediction.

At least one of the number or types of transform candidates is differentfor each transform set. At least one of the number or types of transformcandidates may be variably determined based on at least one of thelocation, the size, the partition form, and the prediction mode(intra/inter mode) or direction/non-direction of the intra-predictionmode of a block (CU, PU, TU, etc.).

The decoder may perform a secondary inverse transform depending onwhether or not the secondary inverse transform is performed, and mayperform a primary inverse transform depending on whether or not theprimary inverse transform is performed from the result of the secondaryinverse transform.

The above-described primary transform and secondary transform may beapplied to at least one signal component of luma/chroma components ormay be applied according to the size/shape of an arbitrary coding block.Entropy encoding/decoding may be performed on an index indicating bothwhether or not the primary transform/secondary transform is used and theused primary transform/secondary transform in an arbitrary coding block.Alternatively, the index may be tacitly derived by the encoder/decoderaccording to at least one piece of current/neighboring encodinginformation.

The residual signal generated after intra or inter prediction goesthrough a quantization process after the primary and/or secondarytransform, and quantized transform coefficients go through an entropyencoding process. Here, the quantized transform coefficients may bescanned in diagonal, vertical, and horizontal directions based on atleast one of the intra-prediction mode or the size/shape of a minimumblock as shown in FIG. 10 .

In addition, the quantized transform coefficients on which entropydecoding is performed may be arranged in block forms by being inversescanned, and at least one of dequantization or inverse transform may beperformed on the relevant block. Here, as a method of inverse scanning,at least one of diagonal direction scanning, horizontal directionscanning, and vertical direction scanning may be performed.

For example, when the size of a current coding block is 8×8, primarytransform, secondary transform, and quantization may be performed on aresidual signal for the 8×8 block, and next, scanning and entropyencoding may be performed on quantized transform coefficients for eachof four 4×4 sub-blocks according to at least one of three scanning ordermethods shown in FIG. 10 . In addition, inverse scanning may beperformed on the quantized transform coefficients by performing entropydecoding. The quantized transform coefficients on which inverse scanningis performed become transform coefficients after dequantization, and atleast one of secondary inverse transform or primary inverse transform isperformed, whereby a reconstructed residual signal can be generated.

In a video encoding process, one block may be partitioned as shown inFIG. 11 , and an indicator corresponding to partition information may besignaled. Here, the partition information may be at least one of apartition flag (split_flag), a quad/binary tree flag (QB_flag), a quadtree partition flag (quadtree_flag), a binary tree partition flag(binarytree_flag), and a binary tree partition type flag (Btype_flag).Here, split_flag is a flag indicating whether or not a block ispartitioned, QB_flag is a flag indicating whether a block is partitionedin a quad tree form or in a binary tree form, quadtree_flag is a flagindicating whether or not a block is partitioned in a quad tree form,binarytree_flag is a flag indicating whether or not a block ispartitioned in a binary tree form, Btype_flag is a flag indicatingwhether a block is vertically or horizontally partitioned in a case ofpartition of a binary tree form.

When the partition flag is 1, it may indicate partitioning is performed,and when the partition flag is 0, it may indicate partitioning is notperformed. In a case of the quad/binary tree flag, 0 may indicate a quadtree partition, and 1 may indicate a binary tree partition.Alternatively, 0 may indicate a binary tree partition, and 1 mayindicate a quad tree partition. In a case of the binary tree partitiontype flag, 0 may indicate a horizontal direction partition, and 1 mayindicate a vertical direction partition. Alternatively, 0 may indicate avertical direction partition, and 1 may indicate a horizontal directionpartition.

For example, partition information for FIG. 11 may be derived bysignaling at least one of quadtree_flag, binarytree_flag, and Btype_flagas shown in table 3.

TABLE 3 quad 1 0 1 0 0 0 0 0 0 tree_flag binary 1 0 0 1 0 0 0 0 0 1 1 00 0 0 tree_flag Btype_flag 1 0 0 1

For example, partition information for FIG. 11 may be derived bysignaling at least one of split_flag, QB_flag, and Btype_flag as shownin table 2.

TABLE 4 split_flag 1 1 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 QB_flag 0 1 0 1 1Btype_flag 1 0 0 1

The partition method may be performed only in a quad tree form or onlyin a binary tree form according to the size/shape of a block. In thiscase, the split_flag may mean a flag indicating whether partitioning isperformed in a quad tree for or in a binary tree form. The size/shape ofa block may be derived according to depth information of a block, andthe depth information may be signaled.

When the size of a block is in a predetermined range, partitioning maybe performed only in a quad tree form. Here, the predetermined range maybe defined as at least one of the size of a maximum block or the size ofa minimum block that can be partitioned only in a quad tree form.Information indicating the size of a maximum/minimum block where apartition in the quad tree form is allowed may be signaled through abitstream, and the information may be signaled by a unit of at least oneof a sequence, a picture parameter, or a slice (segment). Alternatively,the size of a maximum/minimum block may be a fixed size that is presetin the encoder/decoder. For example, when the size of a block ranges256×256 to 64×64, partitioning may be performed only in a quad treeform. In this case, the split_flag may be a flag indicating whetherpartitioning is performed in a quad tree form.

When the size of a block is in a predetermined range, partitioning maybe performed only in a binary tree form. Here, the predetermined rangemay be defined as at least one of the size of a maximum block or thesize of a minimum block that can be partitioned only in a binary treeform. Information indicating the size of a maximum/minimum block where apartition in the binary tree form is allowed may be signaled through abitstream, and the information may be signaled by a unit of at least oneof a sequence, a picture parameter, or a slice (segment). Alternatively,the size of a maximum/minimum block may be a fixed size that is presetin the encoder/decoder. For example, when the size of a block ranges16×16 to 8×8, partitioning may be performed only in a binary tree form.In this case, the split_flag may be a flag indicating whetherpartitioning is performed in a binary tree form.

After partitioning one block in a binary tree form, when the partitionedblock is further partitioned, partitioning may be performed only in abinary tree form.

When the width or length size of the partitioned block cannot be furtherpartitioned, at least one indicator may not be signaled.

Besides the quad tree based binary tree partitioning, the quad treebased partitioning may be performed after the binary tree partitioning.

Based on the above-described description, a method of encoding/decodinga view according to the present invention will be described in detail.

FIG. 12 is a flowchart showing a method for encoding a video by using amerge mode according to the present invention. FIG. 13 is a flowchartshowing a method for decoding a video by using a merge mode according tothe present invention.

Referring to FIG. 12 , an encoding apparatus may derive a mergecandidate at step S1201, and may generate a merge candidate list basedon the derived merge candidate. When the merge candidate list isgenerated, motion information is determined by using the generated mergecandidate list at step S1202, and motion compensation of the currentblock may be performed by using the determined motion information atstep S1203. Next, the encoding apparatus may entropy encode informationon motion compensation at step S1204.

Referring to FIG. 13 , a decoding apparatus may entropy decode theinformation on motion compensation received from the encoding apparatusat step S1301, and may derive the merge candidate at step S1302, and maygenerate the merge candidate list based on the derived merge candidate.When the merge candidate list is generated, motion information of thecurrent block may be determined by using the generated merge candidatelist at step S1303. Next, the decoding apparatus may perform motioncompensation by using the motion information at step S1304.

Hereinafter, steps shown in FIGS. 12 and 13 will be described in detail.

First, deriving of the merge candidate at steps S1201 and S1302 will bedescribed in detail.

The merge candidate for the current block may include at least one of aspatial merge candidate, a temporal merge candidate, and an additionalmerge candidate.

The spatial merge candidate of the current block may be derived from areconstructed block neighboring the current block. For example, motioninformation of the reconstructed block neighboring the current block maybe determined as the spatial merge candidate for the current block.Here, the motion information may include at least one of a motionvector, a reference picture index, and a prediction list utilizationflag.

In this case, the motion information of the spatial merge candidate mayinclude motion information corresponding to L0 and L1 as well as motioninformation corresponding to L0 L1, . . . , LX. Here, X may be apositive integer including zero. Accordingly, a reference picture listmay include at least one of L0 L1, . . . , LX.

FIG. 14 is a view showing an example of deriving a spatial mergecandidate of a current block. Here, the deriving of the spatial mergecandidate may mean deriving the spatial merge candidate and adding thespatial merge candidate to the merge candidate list.

Referring to FIG. 14 , the spatial merge candidate of the current blockmay be derived from neighbor blocks adjacent to the current block X. Theneighbor block adjacent to the current block may include at least one ofa block (B1) adjacent to the top of the current block, a block (A1)adjacent to the left of the current block, a block (B0) adjacent to thetop right corner of the current block, a block (B2) adjacent to the topleft corner of the current block, and a block (A0) adjacent to thebottom left corner of the current block. In the meantime, the neighborblock adjacent to the current block may have a square shape or anon-square shape.

In order to derive the spatial merge candidate of the current block,whether or not the neighbor block adjacent to the current block can beused in deriving the spatial merge candidate of the current block may bedetermined. Here, whether or not the neighbor block adjacent to thecurrent block can be used in deriving the spatial merge candidate of thecurrent block may be determined according to predetermined priorities.For example, in the example shown in FIG. 14 , availability of derivingthe spatial merge candidate may be determined in the order of blocks atA1, B1, B0, A0, and B2 positions. The spatial merge candidatesdetermined based on the order for determining availability may be addedto the merge candidate list of the current block in sequence. Thefollowing are examples of the neighbor block that is unable to be usedin deriving the spatial merge candidate of the current block.

1) a condition where spatial merge candidates are derived from blocks atA0, A1, B0, and B1 positions when the neighbor block is a block at B2position

2) a condition where the neighbor block is absent (a condition where thecurrent block exists at a picture boundary, a slice boundary or a tileboundary, etc.)

3) a condition where the neighbor block is intra coded

4) a condition where at least one of a motion vector, a referencepicture index, and a reference picture of the neighbor block is the sameas that of the previously derived spatial merge candidate

5) a condition where a motion vector of the neighbor block refers to aboundary outer area of at least one of a picture, a slice, and a tile inwhich the current block is included

FIG. 15 is a view showing an example of adding a spatial merge candidateto a merge candidate list.

Referring to FIG. 15 , when four spatial merge candidates are derivedfrom the neighbor blocks at A1, B0, A0, and B2 positions, the derivedspatial merge candidates may be added to the merge candidate list insequence.

maxNumSpatialMergeCand may mean the maximum number of spatial mergecandidates that can be included in the merge candidate list, andnumMergeCand may mean the number of merge candidates included in themerge candidate list. maxNumSpatialMergeCand may be a positive integerincluding zero. maxNumSpatialMergeCand may be preset for the encodingapparatus and the decoding apparatus to use the same value.Alternatively, the encoding apparatus may encode the maximum number ofmerge candidates that can be included in the merge candidate list of thecurrent block, and the maximum number may be signaled to the decodingapparatus through a bitstream.

As described above, when at least one spatial merge candidate is derivedfrom the neighbor blocks A1, B1, B0, A0, and B2, spatial merge candidateflag information (spatialCand) indicating whether it is a spatial mergecandidate may be set for each derived merge candidate. For example, whenthe spatial merge candidate is derived, spatialCand may be set to apredetermined value of one. Otherwise, spatialCand may be set to apredetermined value of zero. Also, whenever the spatial merge candidateis derived, spatial merge candidate count (spatialCandCnt) increases byone.

The spatial merge candidate may be derived based on at least one ofcoding parameters of the current block or the neighbor block.

The spatial merge candidate may be shared in blocks that are smallerthan a block in size or are deeper than the block in depth, the block inwhich information on motion compensation is entropy encoded/decoded.Here, the information on motion compensation may be at least one ofinformation on whether or not a skip mode is used, information onwhether or not a merge mode is used or merge index information.

The block in which the information on motion compensation is entropyencoded/decoded may be a CTU or a sub-unit of a CTU, a CU, or a PU.

Hereinafter, for convenience of explanation, the size of the block inwhich the information on motion compensation is entropy encoded/decodedis referred to as a first block size. The depth of the block in whichthe information on motion compensation is entropy encoded/decoded isreferred to as a first block depth.

Specifically, when the size of the current block is smaller than thefirst block size, the spatial merge candidate of the current block maybe derived from at least one of reconstructed blocks adjacent to thehigher block having the first block size. Also, the blocks included inthe higher block may share the derived spatial merge candidate. Here,the block having the first block size may be referred to as a higherblock of the current block.

FIG. 16 is a view showing an embodiment of deriving and sharing aspatial merge candidate in a CTU. Referring to FIG. 16 , when the firstblock size is 32×32, the blocks 1601, 1602, 1603, and 1604 that aresmaller than 32×32 may derive a spatial merge candidate from at leastone of neighbor blocks adjacent to the higher block 1600 having thefirst block size, and may share the derived spatial merge candidate.

For example, when the first block size is 32×32 and the size of thecoding block is 32×32, prediction blocks that are smaller than 32×32 mayderive a spatial merge candidate of the prediction block from at leastone piece of motion information of neighbor blocks of the coding block.The prediction blocks in the coding block may share the derived spatialmerge candidate. Here, the coding block and the prediction block maymean blocks which are more generalized expressions.

When the depth of the current block is deeper than the first blockdepth, the spatial merge candidate may be derived from at least one ofreconstructed blocks adjacent to the higher block having the first blockdepth. Also, the blocks included in the higher block may share thederived spatial merge candidate. Here, the block having the first blockdepth may be referred to as a higher block of the current block.

For example, when the first block depth is two and the depth of thecoding block is two, prediction blocks having the depth deeper than theblock depth of two may derive a spatial merge candidate of theprediction block based on at least one piece of motion information ofneighbor blocks of the coding block. The prediction blocks in the codingblock may share the derived spatial merge candidate.

Here, the sharing of the spatial merge candidate may mean thatrespective merge candidate lists of sharing blocks can be generatedbased on the same spatial merge candidate.

Also, the sharing of the spatial merge candidate may mean that sharingblocks can perform motion compensation by using one merge candidatelist. Here, a shared merge candidate list may include at least one ofspatial merge candidates derived based on a higher block in whichinformation on motion compensation is entropy encoded/decoded.

The neighbor block adjacent to the current block or the current blockmay have a square shape or a non-square shape.

Also, the neighbor block adjacent to the current block may bepartitioned into sub-blocks. In this case, among sub-blocks of theneighbor block adjacent to the current block, motion information of onesub-block may be determined as a spatial merge candidate of the currentblock. Also, a spatial merge candidate of a current block may bedetermined based on at least one piece of motion information ofsub-blocks of a neighbor block adjacent to the current block. Here,whether or not the sub-block of the neighbor block can be used inderiving the spatial merge candidate may be decided to determine thespatial merge candidate of the current block. Availability of being usedin deriving the spatial merge candidate may include at least one ofwhether or not motion information of the sub-block of the neighbor blockexists, and whether or not the motion information of the sub-block ofthe neighbor block can be used as the spatial merge candidate of thecurrent block.

Also, one of a median value, an average value, a minimum value, amaximum value, a weighted average value or a mode of at least one piece(i.e., a motion vector) of motion information of the sub-blocks of theneighbor block may be determined as the spatial merge candidate of thecurrent block.

Next, a method of deriving a temporal merge candidate of the currentblock will be described.

The temporal merge candidate of the current block may be derived from areconstructed block included in a collocated picture of a currentpicture. Here, the collocated picture is a picture that has beenencoded/decoded before the current picture. The collocated picture maybe a picture having different temporal order from the current picture.

FIG. 17 is a view showing an example of deriving a temporal mergecandidate of a current block. Here, the deriving of the temporal mergecandidate may mean deriving the temporal merge candidate and adding thetemporal merge candidate to the merge candidate list.

Referring to FIG. 17 , in the collocated picture of the current picture,the temporal merge candidate of the current block may be derived from ablock including an outer position of a block corresponding to spatiallythe same position as the current block X or from a block including aninner position of a block corresponding to spatially the same positionas the current block X. Here, the temporal merge candidate may meanmotion information of the collocated block. For example, the temporalmerge candidate of the current block X may be derived from a block Hadjacent to the bottom right corner of a block C corresponding tospatially the same position as the current block, or from a block C3including the center point of the block C. The block H or the block C3that is used to derive the temporal merge candidate of the current blockmay be referred to as ‘a collocated block’.

In the meantime, the collocated block of the current block or thecurrent block may have a square shape or a non-square shape.

When the temporal merge candidate of the current block can be derivedfrom the block H including the outer position of the block C, the blockH may be set to the collocated block of the current block. In this case,the temporal merge candidate of the current block may be derived basedon motion information of the block H. In contrast, when the temporalmerge candidate of the current block cannot be derived from the block H,the block C3 including the inner position of the block C may be set tothe collocated block of the current block. In this case, the temporalmerge candidate of the current block may be derived based on motioninformation of the block C3. When the temporal merge of the currentblock cannot be derived from the block H and the block C3 (for example,when both the block H and the block C3 are intra coded), the temporalmerge candidate of the current block may not be derived or may bederived from a block at a different position from the block H and theblock C3.

As another example, the temporal merge candidate of the current blockmay be derived from a plurality of blocks in the collocated picture. Forexample, a plurality of temporal merge candidates of the current blockmay be derived from the block H and the block C3.

FIG. 18 is a view showing an example of adding a temporal mergecandidate to a merge candidate list.

Referring to FIG. 18 , when one temporal merge candidate is derived froma collocated block at H1 position, the derived temporal merge candidatemay be added to the merge candidate list.

The collocated block of the current block may be partitioned intosub-blocks. In this case, among the sub-blocks of the collocated blockof the current block, motion information of one sub-block may bedetermined as the temporal merge candidate of the current block. Also,the temporal merge candidate of the current block may be determinedbased on at least one piece of motion information of the sub-blocks ofthe collocated block of the current block.

Here, whether or not motion information of the sub-block of thecollocated block exists or whether or not motion information of thesub-block of the collocated block can be used as the temporal mergecandidate of the current block is decided to determine the temporalmerge candidate of the current block.

Also, one of a median value, an average value, a minimum value, amaximum value, a weighted average value or a mode of at least one piece(i.e., a motion vector) of motion information of sub-blocks of thecollocated block may be determined as the temporal merge candidate ofthe current block.

In FIG. 17 , the temporal merge candidate of the current block can bederived from the block adjacent to the bottom right corner of thecollocated block or from the block including the center point of thecollocated block. However, the positions of the blocks for deriving thetemporal merge candidate of the current block are not limited to theexample shown in FIG. 17 . For example, the temporal merge candidate ofthe current block may be derived from a block adjacent to the top/bottomboundary, the left/right boundary or a corner of the collocated block,and may be derived from a block including a particular position in thecollocated block (i.e., a block adjacent to the corner boundary of thecollocated block).

The temporal merge candidate of the current block may be determined byconsidering the reference picture lists (or prediction direction) of thecurrent block and the collocated block. In the meantime, motioninformation of the temporal merge candidate may include motioninformation corresponding to L0 and L1 as well as motion informationcorresponding to L0, L1, . . . , LX. Here, X may be a positive integerincluding zero.

For example, when the reference picture list that can be used by thecurrent block is L0 (namely, when the inter-prediction indicatorindicates PRED_L0), motion information corresponding to L0 in thecollocated block may be derived as the temporal merge candidate of thecurrent block. That is, when the reference picture list that can be usedby the current block is LX (here, X is an integer such as 0, 1, 2 or 3indicating an index of the reference picture list), motion information(hereinafter, referred to as ‘IX motion information’) corresponding toLX of the collocated block may be derived as the temporal mergecandidate of the current block.

When the current block uses a plurality of reference picture lists, thetemporal merge candidate of the current block may be determined byconsidering the reference picture lists of the current block and thecollocated block.

For example, when bi-directional prediction is performed on the currentblock (namely, the inter-prediction indicator is PRED_BI), at least twopieces of information selected from the group consisting of L0 motioninformation, L1 motion information, L2 motion information, . . . , andLX motion information of the collocated block may be derived as temporalmerge candidates. When tri-directional prediction is performed on thecurrent block (namely, the inter-prediction indicator is PRED_TRI), atleast three pieces of information selected from the group consisting ofL0 motion information, L1 motion information, L2 motion information, . .. , and LX motion information of the collocated block may be derived astemporal merge candidates. When quad-directional prediction is performedon the current block (namely, inter-prediction indicator is PRED_QUAD),at least four pieces of information selected from the group consistingof L0 motion information, L1 motion information, L2 motion information,. . . , and LX motion information of the collocated block may be derivedas temporal merge candidates.

Also, at least one of the temporal merge candidate, the collocatedpicture, the collocated block, the prediction list utilization flag, andthe reference picture index may be derived based on at least one ofcoding parameters of the current block, the neighbor block, and thecollocated block.

When the number of derived spatial merge candidates is less than themaximum number of merge candidates, the temporal merge candidate may bepreliminarily derived. Accordingly, when the number of derived spatialmerge candidates reaches the maximum number of merge candidates, aprocess of deriving the temporal merge candidate may be omitted.

For example, when the maximum number of merge candidates is two and thetwo derived spatial merge candidates have different values, a process ofderiving the temporal merge candidate may be omitted.

As another example, the temporal merge candidate of the current blockmay be derived based on the maximum number of temporal merge candidates.Here, the maximum number of temporal merge candidates may be preset forthe encoding apparatus and the decoding apparatus to use the same value.Alternatively, information indicating the maximum number of temporalmerge candidates of the current block may be encoded through abitstream, and may be signaled to the decoding apparatus. For example,the encoding apparatus may encode maxNumTemporalMergeCand indicating themaximum number of temporal merge candidates of the current block, andmaxNumTemporalMergeCand may be signaled to the decoding apparatusthrough a bitstream. Here, maxNumTemporalMergeCand may be set to apositive integer including zero. For example, maxNumTemporalMergeCandmay be set to one. The value of maxNumTemporalMergeCand may be variouslyderived based on information on the number of temporal merge candidatesbeing signaled, and may be a fixed value preset in the encoder/decoder.

When the distance between the current picture in which the current blockis included and the reference picture of the current block is differentfrom the distance between the collocated picture in which the collocatedblock is included and the reference picture of the collocated block, amotion vector of the temporal merge candidate of the current block maybe obtained by scaling a motion vector of the collocated block. Here,scaling may be performed based on at least one of the distance betweenreference pictures referenced by the current picture and the currentblock, and the distance between reference pictures referenced by thecollocated picture and the collocated block. For example, according tothe ratio of the distance between reference pictures referenced by thecurrent picture and the current block and of the distance betweenreference pictures referenced by the collocated picture and thecollocated block, the motion vector of the collocated block is scaled,thereby deriving the motion vector of the temporal merge candidate ofthe current block.

Based on the size (the first block size) or the depth (the first blockdepth) of a block in which information on motion compensation is entropyencoded/decoded, the temporal merge candidate may be shared in blocksthat are smaller than a block in size or are deeper than the block indepth, the block being the one in which information on motioncompensation is entropy encoded/decoded. Here, the information on motioncompensation may be at least one of information on whether or not a skipmode is used, information on whether or not a merge mode is used, andmerge index information.

The block in which information on motion compensation is entropyencoded/decoded may be a CTU or a sub-unit of a CTU, a CU, or a PU.

Specifically, when the size of the current block is less than the firstblock size, the temporal merge candidate of the current block may bederived from a collocated block of a higher block having the first blocksize. Also, the blocks included in the higher block may share thederived temporal merge candidate.

Also, when the depth of the current block is deeper than the first blockdepth, the temporal merge candidate may be derived from a collocatedblock of a higher block having the first block depth. Also, the blocksincluded in the higher block may share the derived temporal mergecandidate.

Here, the sharing of the temporal merge candidate may mean thatrespective merge candidate lists of sharing blocks can be generatedbased on the same temporal merge candidate.

Also, the sharing of the temporal merge candidate may mean that sharingblocks can perform motion compensation by using one merge candidatelist. Here, shared merge candidate list may include the temporal mergecandidate derived based on a higher block in which information on motioncompensation is entropy encoded/decoded.

FIG. 19 is a view showing an example of scaling a motion vector ofmotion information of a collocated block to derive a temporal mergecandidate of a current block.

The motion vector of the collocated block may be scaled based on atleast one of a difference value (td) between POC (Picture order count)indicating the display order of the collocated picture and POC of thereference picture of the collocated block, and a difference value (tb)between POC of the current picture and POC of the reference picture ofthe current block.

Before performing scaling, td or tb may be adjusted so that td or tbexists within a predetermined range. For example, when the predeterminedrange indicates −128˜127 and td or tb is less than −128, td or tb may beadjusted to −128. When td or tb is greater than 127, td or tb may beadjusted to 127. When td or tb is in the range of −128˜127, td or tb isnot adjusted.

A scaling factor DistScaleFactor may be calculated based on td or tb.Here, the scaling factor may be calculated based on the followingformula 1.

DistScaleFactor=(tb*tx+32)>>6

tx=(16384+Abs(td/2))/td  [Formula 1]

In formula 1, an absolute value function is designated as Abs( ) and theoutput value of the function is the absolute value of the input value.

The value of the scaling factor DistScaleFactor calculated based onformula 1 may be adjusted to a predetermined range. For example,DistScaleFactor may be adjusted to exist in a range of −1024˜1023.

By scaling the motion vector of the collocated block by using thescaling factor, the motion vector of the temporal merge candidate of thecurrent block may be determined. For example, the motion vector of thetemporal merge candidate of the current block may be determined by thefollowing formula 2.

Sign(DistScaleFactor*mvCol)*((Abs(DistScaleFactor*mvCol)+127)>>8)  [Formula2]

In formula 2, Sign( ) is a function outputting sign information of thevalue in ( ). For example, Sign(−1) outputs−. In formula 2, the motionvector of the collocated block may be designated as mvCol.

Next, a method of deriving an additional merge candidate of the currentblock will be described.

The additional merge candidate may mean at least one of a modifiedspatial merge candidate, a modified temporal merge candidate, a combinedmerge candidate, and a merge candidate having a predetermined motioninformation value. Here, the deriving of the additional merge candidatemay mean deriving the additional merge candidate and adding theadditional merge candidate to the merge candidate list.

The modified spatial merge candidate may mean a merge candidate in whichat least one piece of motion information of the derived spatial mergecandidate is modified.

The modified temporal merge candidate may mean a merge candidate inwhich at least one piece of motion information of the derived temporalmerge candidate is modified.

The combined merge candidate may mean a merge candidate derived bycombining at least one piece of motion information among motioninformation of the spatial merge candidate, the temporal mergecandidate, the modified spatial merge candidate, the modified temporalmerge candidate, the combined merge candidate, and the merge candidateshaving the predetermined motion information value that exist in themerge candidate list.

Alternatively, the combined merge candidate may mean a merge candidatederived by combining at least one piece of motion information of aspatial merge candidate, a temporal merge candidate, a modified spatialmerge candidate, a modified temporal merge candidate, a combined mergecandidate, and a merge candidate having a predetermined motioninformation value. The spatial merge candidate and the temporal mergecandidate are derived from a block that does not exist in the mergecandidate list but can be used to derive at least one of the spatialmerge candidate and the temporal merge candidate. The modified spatialmerge candidate and the modified temporal merge candidate are generatedbased on the spatial merge candidate and the temporal merge candidate.

Alternatively, the combined merge candidate may be derived by usingmotion information that is entropy decoded from a bitstream by thedecoder. Here, the motion information used in deriving the combinedmerge candidate may be entropy encoded in a bitstream by the encoder.

The combined merge candidate may mean a combined bi-predictive mergecandidate. The combined bi-predictive merge candidate is a mergecandidate using bi-prediction, and may mean a merge candidate having L0motion information and L1 motion information.

Also, the combined merge candidate may mean a merge candidate having atleast N selected from the group consisting of L0 motion information, L1motion information, L2 motion information, and L3 motion information.Here, N may mean a positive integer equal to or greater than two.

The merge candidate having the predetermined motion information valuemay mean a zero merge candidate of which the motion vector is (0, 0). Inthe meantime, the merge candidate having the predetermined motioninformation value may be preset for the encoding apparatus and thedecoding apparatus to use the same value.

At least one of the modified spatial merge candidate, the modifiedtemporal merge candidate, the combined merge candidate, and the mergecandidate having the predetermined motion information value may bederived or generated based on at least one of coding parameters of thecurrent block, the neighbor block, and the collocated block. Also, atleast one of the modified spatial merge candidate, the modified temporalmerge candidate, the combined merge candidate, and the merge candidatehaving the predetermined motion information value may be added to themerge candidate list based on at least one of the coding parameters ofthe current block, the neighbor block, and the collocated block.

The additional merge candidate may be derived for each sub-block of thecurrent block, the neighbor block, or the collocated block. The mergecandidate derived for each sub-block may be added to the merge candidatelist of the current block.

The additional merge candidate may be derived in only a case of Bslice/B picture or a case of a slice/picture using at least M referencepicture lists. Here, M may be three or four, and may mean a positiveinteger equal to or greater than three.

Up to N additional merge candidates may be derived. Here, N is apositive integer including zero. N may be a variable value derived basedon information on the maximum number of merge candidates included in themerge candidate list. Alternatively, N may be a fixed value preset inthe encoder/decoder. Here, N may differ depending on the size, shape,depth or position of a block encoded/decoded in a merge mode.

The size of the merge candidate list is a preset size, and may beincreased by the number of additional merge candidates generated afteradding the spatial merge candidate or the temporal merge candidate. Inthis case, all the generated additional merge candidates may be includedin the merge candidate list. In contrast, the size of the mergecandidate list may be increased to a size smaller than the number of theadditional merge candidates (for example, the number of the additionalmerge candidates—N, N is a positive integer). In this case, only a partof the generated additional merge candidates may be included in themerge candidate list.

Also, the size of the merge candidate list may be determined based onthe coding parameters of the current block, the neighbor block, or thecollocated block, and may be changed based on the coding parameters.

In order to increase throughput of the merge mode in the encoder and thedecoder, motion compensation using the merge mode may be performedthrough only spatial merge candidate derivation, temporal mergecandidate derivation, and zero merge candidate derivation withoutderiving the combined merge candidate. In a case of performing acombined merge candidate derivation process after a temporal mergecandidate derivation process which requires relatively considerablecycle time, when the combined merge candidate derivation process is notperformed, the worst case of the hardware complexity of the merge modemay be the temporal merge candidate derivation process rather than thecombined merge candidate derivation process after the temporal mergecandidate derivation process. Accordingly, the cycle time required inderiving each merge candidate in the merge mode may be reduced. Also, inthe merge mode where the combined merge candidate is not derived, thereis no dependency between merge candidate derivation processes. Thus, itis advantageous in that spatial merge candidate derivation, temporalmerge candidate derivation, and zero merge mode candidate derivation maybe performed in parallel.

FIG. 21A and FIG. 21B are views showing an embodiment of a method ofderiving a combined merge candidate. When at least one merge candidateexists in the merge candidate list or when the number of the mergecandidates (numOrigMergeCand) in the merge candidate list is less thanthe maximum number of merge candidates (MaxNumMergeCand) before derivingthe combined merge candidate, the method of deriving the combined mergecandidate in FIG. 21 a and FIG. 21 b may be performed.

Referring to FIG. 21A and FIG. 21B, the encoder/decoder may set theinput number of merge candidates (numInputMergeCand) as the number ofmerge candidates (numMergeCand) in the current merge candidate list, andmay set the combined index (combIdx) to zero. The k-th (numMergeCand−numinputMergeCand) combined merge candidate may be derived.

The encoder/decoder may derive at least one of L0 candidate index(I0CandIdx), L1 candidate index (I1CandIdx), L2 candidate index(I2CandIdx), and L3 candidate index (13CandIdx) by using the combinedindexes as shown in FIG. 20 at step S2101.

Each candidate index may indicate the merge candidate in the mergecandidate list. Motion information of the candidate index according toL0 L1, L2, and L3 may be motion information on L0 L1, L2, and L3 of thecombined merge candidate.

The encoder/decoder may derive L0 candidate (I0Cand) as a mergecandidate (mergeCandList[I0CandIdx]) corresponding to L0 candidate indexin the merge candidate list, may derive L1 candidate (I1Cand) as a mergecandidate(mergeCandList[I1CandIdx]) corresponding to L1 candidate indexin the merge candidate list, and may derive L2 candidate (I2Cand) as amerge candidate(mergeCandList[I2CandIdx]) corresponding to L2 candidateindex in the merge candidate list, and may derive L3 candidate (I3Cand)as a merge candidate(mergeCandList[I3CandIdx]) corresponding to L3candidate index in the merge candidate list at step S2102.

The encoder/decoder may perform step S2104 when at least one of thefollowing conditions is satisfied at step S2103 and, otherwise, mayperform step S2105.

1) a condition where L0 candidate uses L0 uni-prediction(predFlagL0I0Cand==1)

2) a condition where L1 candidate uses L1 uni-prediction(predFlagL1I1Cand==1)

3) a condition where L2 candidate uses L2 uni-prediction(predFlagL2I2Cand==1)

4) a condition where L3 candidate uses L3 uni-prediction(predFlagL3I3Cand==1)

5) a condition where at least one reference picture of L0 L1, L2, and L3candidates is different from a reference picture of another candidate,and at least one motion vector of L0 L1, L2, and L3 candidates isdifferent from a motion vector of another candidate

When at least one of the above-mentioned five conditions is satisfied atstep S2103—Yes, the encoder/decoder may determine L0 motion informationof L0 candidate as L0 motion information of a combined candidate, maydetermine L1 motion information of L1 candidate as L1 motion informationof a combined candidate, may determine L2 motion information of L2candidate as L2 motion information of a combined candidate, maydetermine L3 motion information of L3 candidate as L3 motion informationof a combined candidate, and may add a combined merge candidate(combCandk) to the merge candidate list at step S2104.

For example, information on the combined merge candidate may be the sameas follows.

L0 reference picture index of K-th combined merge candidate(refIdxL0combCandk)=L0 reference picture index of L0 candidate(refIdxL0I0Cand)

L1 reference picture index of K-th combined merge candidate(refIdxL1combCandk)=L1 reference picture index of L1 candidate(refIdxL1I1Cand)

L2 reference picture index of K-th combined merge candidate(refIdxL2combCandk)=L2 reference picture index of L2 candidate(refIdxL2I2Cand)

L3 reference picture index of K-th combined merge candidate(refIdxL3combCandk)=L3 reference picture index of L3 candidate(refIdxL3I3Cand)

L0 prediction list utilization flag of K-th combined merge candidate(predFlagL0combCandk)=1

L1 prediction list utilization flag of K-th combined merge candidate(predFlagL1combCandk)=1

L2 prediction list utilization flag of K-th combined merge candidate(predFlagL2combCandk)=1

L3 prediction list utilization flag of K-th combined merge candidate(predFlagL3combCandk)=1

x component of L0 motion vector of K-th combined merge candidate(mvL0combCandk[0])=x component of L0 motion vector of L0 candidate(mvL0I0Cand[0])

y component of L0 motion vector of K-th combined merge candidate(mvL0combCandk[1])=y component of L0 motion vector of L0 candidate(mvL0I0Cand[1])

x component of L1 motion vector of K-th combined merge candidate(mvL1combCandk[0])=x component of L1 motion vector of L1 candidate(mvL1I1Cand[0])

y component of L1 motion vector of K-th combined merge candidate(mvL1combCandk[1])=y component of L1 motion vector of L1 candidate(mvL1I1Cand[1])

x component of L2 motion vector of K-th combined merge candidate(mvL2combCandk[0])=x component of L2 motion vector of L2 candidate(mvL2I2Cand[0])

y component of L2 motion vector of K-th combined merge candidate(mvL2combCandk[1])=y component of L2 motion vector of L2 candidate(mvL2I2Cand[1])

x component of L3 motion vector of K-th combined merge candidate(mvL3combCandk[0])=x component of L3 motion vector of L3 candidate(mvL3I3Cand[0])

y component of L3 motion vector of K-th combined merge candidate(mvL3combCandk[1])=y component of L3 motion vector of L3 candidate(mvL3I3Cand[1]) numMergeCand=numMergeCand+1

Also, the encoder/decoder may increase the combined index by one at stepS2105.

Also, when the combined index is equal to(numOrigMergeCand*(numOrigMergeCand−1)) or when the number of mergecandidates (numMergeCand) in the current merge candidate list is equalto the maximum number of merge candidates (MaxNumMergeCand) at stepS2106, the encoder/decoder may terminate the combined merge candidatederivation step, and other wise, may perform step S2101.

When performing the method of deriving the combined merge candidate inFIG. 21A and FIG. 21B, the derived combined merge candidate may be addedto the merge candidate list as shown in FIG. 22 .

In the meantime, when at least two spatial merge candidates exist in themerge candidate list or when the number of merge candidates(numOrigMergeCand) in the merge candidate list is less than the maximumnumber of merge candidates

(MaxNumMergeCand) before deriving the combined merge candidate, themethod of deriving the combined merge candidate by using only thespatial merge candidates may be performed. In this case, the method ofderiving the combined merge candidate in FIG. 21A and FIG. 21B may beused.

However, L0 candidate index, L1 candidate index, L2 candidate index, andL3 candidate index that are derived at step S2101 on FIG. 21A mayindicate only merge candidates of which spatial merge candidate flaginformation (spatialCand) is one. Accordingly, L0 candidate, L1candidate, L2 candidate, and L3 candidate that are derived at step S2102may be derived by using merge candidates of which spatial mergecandidate flag information (spatialCand) is one in the merge candidatelist, namely, only by using spatial merge candidates.

Also, at step S2106 of FIG. 21B, instead of the value of(numOrigMergeCand*(numOrigMergeCand−1)), the value of(spatialCandCnt*(spatialCandCnt−1)) is compared with the combined index.When the combined index is equal to (spatialCandCnt*(spatialCandCnt−1))or when the number of merge candidates (numMergeCand) in the currentmerge candidate list is equal to ‘MaxNumMergeCand’, the combined mergecandidate derivation step may be terminated and, otherwise, step S2101may be performed.

When performing the method of deriving the combined merge candidate byusing only spatial merge candidates, the combined merge candidatecombined only with spatial merge candidates may be added to the mergecandidate list as shown in FIG. 23 .

FIGS. 22 and 23 are views showing an example of deriving a combinedmerge candidate by using at least one of a spatial merge candidate, atemporal merge candidate, and a zero merge candidate, and of adding thecombined merge candidate to a merge candidate list.

Here, a merge candidate having at least one piece of L0 motioninformation, L1 motion information, L2 motion information, and L3 motioninformation may be included in the merge candidate list. In themeantime, L0 L1, L2, L3 reference picture lists have been described asexamples, without being limited thereto. A merge candidate having motioninformation on L0 LX reference picture lists (X is a positive integer)may be included in the merge candidate list.

Each piece of motion information may include at least one of a motionvector, a reference picture index, and a prediction list utilizationflag.

As shown in FIGS. 22 and 23 , at least one of merge candidates may bedetermined as the final merge candidate. The determined final mergecandidate may be used as motion information of the current block. Themotion information may be used in inter prediction or motioncompensation of the current block. Also, by changing at least one valueof information corresponding to motion information of the current block,the motion information may be used in inter prediction or motioncompensation of the current block. Here, the value to be changed amonginformation corresponding to motion information may be at least one of xcomponent of the motion vector, y component of the motion vector, andthe reference picture index. Also, when changing at least one value ofinformation corresponding to motion information, at least one value ofinformation corresponding to motion information may be changed so as toindicate the minimum distortion by using a distortion calculation method(SAD, SSE, MSE, etc.).

The prediction block for the current block may be generated by using atleast one piece of L0 motion information, L1 motion information, L2motion information, and L3 motion information according to motioninformation of the merge candidate. The generated prediction block maybe used in inter prediction or motion compensation of the current block.

When at least one piece of L0 motion information, L1 motion information,L2 motion information, and L3 motion information is used in generatingthe prediction block, the inter-prediction indicator may be indicated asPRED_LX which is uni-directional prediction indicating PRED_L0 orPRED_L1, and as PRED_BI_LX which is bi-directional prediction for areference picture list X. Here, X may mean a positive integer including0 such as 0, 1, 2, 3, etc.

Also, the inter-prediction indicator may be indicated as PRED_TRI whichis tri-directional prediction when at least three pieces of informationselected from the group consisting of L0 motion information, L1 motioninformation, L2 motion information, and L3 motion information are used.Also, the inter-prediction indicator may be indicated as PRED_QUAD whichis quad-directional prediction when at least four pieces of informationselected from the group consisting of L0 motion information, L1 motioninformation, L2 motion information, and L3 motion information are used.

For example, when the inter-prediction indicator for the referencepicture list L0 is PRED_L0 and the inter-prediction indicator for thereference picture list L1 is PRED_BI_L1, the inter-prediction indicatorof the current block may be PRED_TRI. That is, the sum of the number ofprediction blocks indicated by the inter-prediction indicator for eachreference picture list may be the inter-prediction indicator of thecurrent block.

Also, there may be at least one reference picture list such as L0 L1,L2, L3, etc. The merge candidate list as shown in FIGS. 22 and 23 may begenerated for each reference picture list. Accordingly, when generatingthe prediction block for the current block, at least one to at most Nprediction blocks may be generated to be used in inter prediction ormotion compensation for the current block. Here, N may mean a positiveinteger equal to or greater than 1 such as 1, 2, 3, 4, etc.

In order to reduce memory bandwidth and to enhance processing speed,when at least one of the reference picture index and the motion vectorof the merge candidate is the same as that of another merge candidate oris in a predetermined range, it may be used in deriving the combinedmerge candidate.

For example, among the merge candidates included in the merge candidatelist, the merge candidates of which the reference picture indexes arethe same at a predetermined value may be used in deriving the combinedmerge candidate. Here, the predetermined value may be a positive integerincluding zero.

As another example, among the merge candidates included in the mergecandidate list, the merge candidates of which the reference pictureindexes are in a predetermined range may be used in deriving thecombined merge candidate. Here, the predetermined range may be a rangeof a positive integer including zero.

As another example, among the merge candidates included in the mergecandidate list, the merge candidates of which the motion vectors are ina predetermined range may be used in deriving the combined mergecandidate. Here, the predetermined range may be a range of a positiveinteger including zero.

As another example, among the merge candidates included in the mergecandidate list, the merge candidates of which motion vector differencevalues between the merge candidates are in a predetermined range may beused in deriving the combined merge candidate. Here, the predeterminedrange may be a range of a positive integer including zero.

Here, at least one of the predetermined value and predetermined rangemay be determined based on a value set in the encoder/decoder in common.Also, at least one of the predetermined value and predetermined rangemay be determined based on a value that is entropy encoded/decoded.

Also, in deriving the modified spatial merge candidate, the modifiedtemporal merge candidate, and the merge candidate having thepredetermined motion information value, when at least one of thereference picture index and the motion vector of the merge candidate isthe same as that of another merge candidate or is in a predeterminedrange, it may be used in deriving the combined merge candidate, that maybe derived and added to the merge candidate list.

FIG. 24 is a view showing an advantage of deriving a combined mergecandidate by using only spatial merge candidates in motion compensationusing a merge mode.

Referring to FIG. 24 , in order to increase throughput of the merge modein the encoder and the decoder, the combined merge candidate may bederived by using only the spatial merge candidates without using thetemporal merge candidate. Compared to the spatial merge candidatederivation process, the temporal merge candidate derivation processrequires relatively considerable cycle time due to motion vectorscaling. Accordingly, in a case of performing the combined mergecandidate derivation process after the temporal merge candidatederivation process, when motion information is determined by using themerge mode, considerable cycle time is required.

However, when deriving the combined merge candidate by using only thespatial merge candidates without using the temporal merge candidate, thecombined merge candidate derivation process may be performed immediatelyafter the spatial merge candidate derivation process which requiresrelatively less cycle time, compared to the temporal merge candidatederivation process. Thus, required cycle time may be reduced whendetermining motion information by using the merge mode, compared to amethod including the temporal merge candidate derivation process.

That is, throughput of the merge mode may be enhanced by removingdependency between the temporal merge candidate derivation process andthe combined merge candidate derivation process. Also, when errors occurin the reference picture due to transmission error, etc., the combinedmerge candidate may be derived by using only the spatial mergecandidates instead of the temporal merge candidate, whereby errorresiliency of the decoder may be enhanced.

Also, when using a method of deriving the combined merge candidate byusing only the spatial merge candidates without using the temporal mergecandidate, a method of deriving the combined merge candidate by usingthe temporal merge candidate and a method of deriving the combined mergecandidate without using the temporal merge candidate may be performed inthe same manner. The method may be realized in the same manner, and thushardware logic may be integrated.

FIG. 25 is a view showing an embodiment of a method of partitioning acombined bi-predictive merge candidate. Here, the combined bi-predictivemerge candidate may be a combined merge candidate including two piecesof L0 motion information, . . . , LX motion information. Hereinafter,FIG. 25 will be described on the assumption that the combinedbi-predictive merge candidate includes L0 motion information and L1motion information.

Referring to FIG. 25 , the encoder/decoder may add L0 motion informationand L1 motion information being partitioned from information of thecombined bi-predictive merge candidate in the merge candidate list, tothe merge candidate list as new merge candidate.

Specifically, the encoder/decoder may determine the combinedbi-predictive merge candidate on which partitioning is performed in themerge candidate list by using a partition index (splitIdx) at stepS2501. Here, the partition index (splitIdx) may be index informationindicating a combined bi-predictive merge candidate on whichpartitioning is performed.

The encoder/decoder may set L0 motion information of the combinedbi-predictive merge candidate as motion information of L0 partitioncandidate, and may add the motion information to the merge candidatelist, and may increase numMergeCand by one at step S2502.

The encoder/decoder may determine whether or not the number of mergecandidates (numMergeCand) in the current merge candidate list is thesame as the maximum number of merge candidates (MaxNumMergeCand). Whenthey are the same at step S2503—Yes, the partitioning process may beterminated. In contrast, when they are different from each other at stepS2503-No, the encoder/decoder may set L1 motion information of thecombined bi-predictive merge candidate as motion information of L1partition candidate, and may add the motion information to the mergecandidate list, and may increase numMergeCand by one at step S2504.Next, the partition index (splitIdx) may be increased by one at stepS2505.

Also, when the number of merge candidates (numMergeCand) in the currentmerge candidate list is equal to the maximum number of merge candidates(MaxNumMergeCand) at step S2506, the encoder/decoder may terminate thecombined merge candidate partitioning process and, otherwise, step S2501may be performed.

The method of partitioning the combined bi-predictive merge candidate,which partitions a bi-directional merge candidate, as shown in FIG. 25may be performed only in a case of B slice/B picture or in a case of aslice/picture using at least M reference picture lists. Here, M may bethree or four, and may mean a positive integer equal to or greater thanthree.

The partitioning of the combined bi-predictive merge candidate may beperformed by using at least one of 1) a method of partitioning thecombined bi-predictive merge candidate into uni-prediction mergecandidates when the combined bi-predictive merge candidate exits, 2) amethod of partitioning the combined bi-predictive merge candidate intouni-prediction merge candidates when the combined bi-predictive mergecandidate exists and L0 reference picture and L1 reference picture aredifferent from each other in the combined bi-predictive merge candidate,and 3) a method of partitioning the combined bi-predictive mergecandidate into uni-prediction merge candidates when the combinedbi-predictive merge candidate exists and L0 reference picture and L1reference picture are the same in the combined bi-predictive mergecandidate.

The combined bi-predictive merge candidate uses bi-prediction and motioncompensation is performed by using reconstructed pixel data in at mosttwo different reference pictures, and thus memory access bandwidth inmotion compensation is large, compared to uni-prediction usingreconstructed pixel data in one reference picture. Accordingly, whenusing partitioning of the combined bi-predictive merge candidate, thecombined bi-predictive merge candidate is partitioned intouni-prediction merge candidates. Thus, when the partitioneduni-prediction merge candidate is determined as motion information ofthe current block, memory access bandwidth in motion compensation may bereduced.

The encoder/decoder may derive a zero merge candidate having a zeromotion vector where a motion vector is (0, 0).

The zero merge candidate may mean a merge candidate of which a motionvector of at least one piece of L0 motion information, L1 motioninformation, L2 motion information, and L3 motion information is (0, 0).

Also, the zero merge candidate may be at least one of two types. Thefirst zero merge candidate may mean a merge candidate of which a motionvector is (0, 0) and a reference picture index has a value equal to orgreater than zero. The second zero merge candidate may mean a mergecandidate of which a motion vector is (0, 0) and a reference pictureindex has a value only zero.

When the number of merge candidates (numMergeCand) in the current mergecandidate list is different from the maximum number of merge candidates(MaxNumMergeCand) (namely, when the merge candidate list is not full ofmerge candidates), at least one of the first zero merge candidate andthe second zero merge candidate may be repeatedly added to the mergecandidate list until the number of merge candidates (numMergeCand) isequal to the maximum number of merge candidates (MaxNumMergeCand).

Also, the first zero merge candidate may be derived and added to themerge candidate list. The second zero merge candidate may be derived andadded to the merge candidate list when the merge candidate list is notfull of merge candidates.

FIG. 26 is a view showing an embodiment of a method of deriving a zeromerge candidate. When the number of merge candidates (numMergeCand) inthe current merge candidate list is less than the maximum number ofmerge candidates (MaxNumMergeCand), the deriving of the zero mergecandidate may be performed in the same order as shown in FIG. 26 .

First, the encoder/decoder may set the input number of merge candidates(numInputMergeCand) as the number of merge candidates (numMergeCand) inthe current merge candidate list. Also, a reference picture index(zeroIdx) of the zero merge candidate may be set to zero. Here, m-th(numMergeCand −numInputMergeCand) zero merge candidate may be derived.

The encoder/decoder may determine whether or not the slice type(slice_type) is P slice at step S2601.

When the slice type (slice_type) is P slice at step S2601—Yes, theencoder/decoder may set the number of reference pictures (numRefIdx) tothe number of available reference pictures in L0 list(num_ref_idx_I0_active_minus1+1).

Also, the encoder/decoder may derive the zero merge candidate as followsand may increase numMergeCand by one at step S2602.

reference picture index of m-th zero merge candidate(refIdxL0zeroCandm)=reference picture index of the zero merge candidate(zeroIdx)

L1 reference picture index of m-th zero merge candidate(refIdxL1zeroCandm)=−1

L0 prediction list utilization flag of m-th zero merge candidate(predFlagL0zeroCandm)=1

L1 prediction list utilization flag of m-th zero merge candidate(predFlagL1 zeroCandm)=0

x component of L0 motion vector of m-th zero merge candidate(mvL0zeroCandm[0])=0

y component of L0 motion vector of m-th zero merge candidate(mvL0zeroCandm[1])=0

x component of L1 motion vector of m-th zero merge candidate(mvL1zeroCandm[0])=0

y component of L1 motion vector of m-th zero merge candidate(mvL1zeroCandm[1])=0

In contrast, when the slice type is not P slice (is B slice or anotherslice) at step S2601-No, the number of reference pictures (numRefIdx)may be set to a value less than at least one of the number of availablereference pictures in L0 list (num_ref_idx_I0_active_minus1+1), thenumber of available reference pictures in L1 list(num_ref_idx_I1_active_minus1+1), the number of available referencepictures in L2 list (num_ref_idx_I2_active_minus1+1), and the number ofavailable reference pictures in L3 list(num_ref_idx_I3_active_minus1+1).

Next, the encoder/decoder may derive the zero merge candidate as followsand may increase numMergeCand by one at step S2603.

refIdxL0zeroCandm=zeroIdx

refIdxL1zeroCandm=zeroIdx

refIdxL2zeroCandm=zeroIdx

refIdxL3zeroCandm=zeroIdx

predFlagL0zeroCandm=1

predFlagL1zeroCandm=1

predFlagL2zeroCandm=1

predFlagL3zeroCandm=1

mvL0zeroCandm[0]=0

mvL0zeroCandm[1]=0

mvL1zeroCandm[0]=0

mvL1zeroCandm[1]=0

mvL2zeroCandm[0]=0

mvL2zeroCandm[1]=0

mvL3zeroCandm[0]=0

mvL3zeroCandm[1]=0

After performing step S2602 or step S2603, when reference picture count(refCnt) is equal to the number of reference pictures (numRefIdx)−1, theencoder/decoder may set the reference picture index of the zero mergecandidate (zeroIdx) to zero and, otherwise, may increase refCnt andzeroIdx by one at step S2604.

Next, when numMergeCand is equal to MaxNumMergeCand at step S2605, theencoder/decoder may terminate the zero merge candidate derivationprocess and, otherwise, step S2601 may be performed.

When the method of deriving the zero merge candidate in FIG. 26 isperformed, the derived zero merge candidate may be added to the mergecandidate list as shown in FIG. 27 .

FIG. 28 is a view showing another embodiment of a method of deriving azero merge candidate. When the number of merge candidates (numMergeCand)in the current merge candidate list is less than the maximum number ofmerge candidates (MaxNumMergeCand), the deriving of L0 uni-predictionzero merge candidate may be performed in the same order as shown in FIG.28 .

First, the encoder/decoder may set the input number of merge candidates(numInputMergeCand) as the number of merge candidates (numMergeCand) inthe current merge candidate list. Also, the reference picture index ofthe zero merge candidate (zeroIdx) may be set to zero. Here, m-th(numMergeCand−numInputMergeCand) zero merge candidate may be derived.Also, the number of reference pictures (numRefIdx) may be set to thenumber of available reference pictures in L0 list(num_ref_idx_I0_active_minus1+1).

The encoder/decoder may derive the zero merge candidate as follows andmay increase numMergeCand by one at step S2801.

L0 reference picture index of m-th zero merge candidate(refIdxL0zeroCandm)=reference picture index of a zero merge candidate(zeroIdx)

L1 reference picture index of m-th zero merge candidate(refIdxL1zeroCandm)=−1

L0 prediction list utilization flag of m-th zero merge candidate(predFlagL0zeroCandm)=1

L1 prediction list utilization flag of m-th zero merge candidate(predFlagL1 zeroCandm)=0

x component of L0 motion vector of m-th zero merge candidate(mvL0zeroCandm[0])=0

y component of L0 motion vector of m-th zero merge candidate(mvL0zeroCandm[1])=0

x component of L1 motion vector of m-th zero merge candidate(mvL1zeroCandm[0])=0

y component of L1 motion vector of m-th zero merge candidate(mvL1zeroCandm[1])=0

When reference picture count (refCnt) is equal to numRefIdx−1, theencoder/decoder may set zeroIdx to zero and, otherwise, refCnt andzeroIdx may be increased by one at step S2802.

Next, when numMergeCand is equal to MaxNumMergeCand at step S2803—Yes,the encoder/decoder may terminate the zero merge candidate derivationstep, and otherwise at step S2803-No, step S2801 may be performed.

In the method of deriving the zero merge candidate in FIG. 26 ,bi-prediction zero merge candidate derivation or L0 uni-prediction zeromerge candidate derivation is performed depending on the slice type.Thus, two realization methods are required depending on the slice type.

In the method of deriving the zero merge candidate in FIG. 28 ,bi-prediction zero merge candidate derivation or L0 uni-prediction zeromerge candidate derivation is not performed depending on the slice type.L0 uni-prediction zero merge candidate is derived regardless of theslice type, whereby hardware logic can be simple. Also, cycle timerequired in deriving the zero merge candidate can be reduced. Also,rather than a bi-prediction zero merge candidate, when the L0uni-prediction zero merge candidate is determined as motion informationof the current block, uni-prediction motion compensation is performedrather than bi-prediction motion compensation. Thus, memory accessbandwidth can be reduced in motion compensation.

For example, except for a case of P slice, the L0 uni-prediction zeromerge candidate may be derived and added to the merge candidate list.

The encoder/decoder may add other merge candidates except for the zeromerge candidate to the merge candidate list, and then may add the L0uni-prediction zero merge candidate. Also, the encoder/decoder mayinitialize the merge candidate list with the L0 uni-prediction zeromerge candidate, and then may add the spatial merge candidate, thetemporal merge candidate, the combined merge candidate, the zero mergecandidate, the additional merge candidate, etc. to the initialized mergecandidate list.

FIG. 29 is a view showing an embodiment of deriving and sharing a mergecandidate list in a CTU. The merge candidate list may be shared inblocks that are smaller than a predetermined block in size or are deeperthan the predetermined block in depth. Here, the size or the depth ofthe predetermined block may be the size or the depth of a block in whichinformation on motion compensation is entropy encoded/decoded. Also, thesize or the depth of the predetermined block may be information that isentropy encoded in the encoder and is entropy decoded in the decoder,and may be a preset value in the encoder/decoder in common.

Referring to FIG. 29 , when the size of the predetermined block is128×128, blocks smaller than 128×128 in size (blocks in the slashed areain FIG. 29 ) may share the merge candidate list.

Next, the determining of motion information of the current block byusing the generated merge candidate list at steps S1202 and S1303 willbe described in detail.

The encoder may determine a merge candidate being used in motioncompensation among merge candidates in the merge candidate list throughmotion estimation, and may encode a merge candidate index (merge_idx)indicating the determined merge candidate in a bitstream.

In the meantime, in order to generate the prediction block, the encodermay select a merge candidate from the merge candidate list based on themerge candidate index to determine motion information of the currentblock. Here, the prediction block of the current block may be generatedby performing motion compensation based on the determined motioninformation.

For example, when the merge candidate index is three, a merge candidateof the merge candidate list indicated by the merge candidate index 3 maybe determined as motion information, and may be used in motioncompensation of an encoding target block.

The decoder may decode the merge candidate index in the bitstream todetermine a merge candidate of the merge candidate list indicated by themerge candidate index. The determined merge candidate may be determinedas motion information of the current block. The determined motioninformation may be used in motion compensation of the current block.Here, motion compensation may mean inter prediction.

For example, when the merge candidate index is two, a merge candidate ofthe merge candidate list indicated by the merge candidate index 2 may bedetermined as motion information, and may be used in motion compensationof a decoding target block.

Also, by changing at least one value of information corresponding tomotion information of the current block, the motion information may beused in inter prediction or motion compensation of the current block.Here, the changed value of information corresponding to motioninformation may be at least one of x component of the motion vector, ycomponent of the motion vector, and the reference picture index. Also,when changing at least one value of information corresponding to motioninformation, at least one value of information corresponding to motioninformation may be changed so as to indicate the minimum distortion byusing a distortion calculation method (SAD, SSE, MSE, etc.).

Next, the performing of motion compensation of the current block byusing the determined motion information at steps S1203 and S1304 will bedescribed in detail.

The encoder and decoder may perform inter prediction or motioncompensation by using motion information of the determined mergecandidate. Here, the current block (encoding/decoding target block) mayhave motion information of the determined merge candidate.

The current block may have at least one to at most N pieces of motioninformation according to prediction direction. At least one to at most Nprediction blocks may be generated by using motion information to derivethe final prediction block of the current block.

For example, when the current block has one piece of motion information,the prediction block generated by using the motion information may bedetermined as the final prediction block of the current block.

In contrast, when the current block has several pieces of motioninformation, several prediction blocks may be generated by using theseveral pieces of motion information, and the final prediction block ofthe current block may be determined based on a weighted sum of theseveral prediction blocks. Reference pictures respectively including theseveral prediction blocks indicated by several pieces of motioninformation may be included in different reference picture lists, andmay be included in the same reference picture list. Also, when thecurrent block has several pieces of motion information, several piecesof motion information of several reference pictures may indicate thesame reference picture.

For example, a plurality of prediction blocks may be generated based onat least one of the spatial merge candidate, the temporal mergecandidate, the modified spatial merge candidate, the modified temporalmerge candidate, the merge candidate having the predetermined motioninformation value or the combined merge candidate, and the additionalmerge candidate. The final prediction block of the current block may bedetermined based on the weighted sum of the plurality of predictionblocks.

As another example, a plurality of prediction blocks may be generatedbased on merge candidates indicated by a preset merge candidate index.The final prediction block of the current block may be determined basedon the weighted sum of the plurality of prediction blocks. Also, aplurality of prediction blocks may be generated based on mergecandidates that exist in a preset merge candidate index range. The finalprediction block of the current block may be determined based on theweighted sum of the plurality of prediction blocks.

The weighting factor applied to each prediction block may have the samevalue by 1/N (here, N is the number of generated prediction blocks). Forexample, when two prediction blocks are generated, the weighting factorbeing applied to each prediction block may be ½. When three predictionblocks are generated, the weighting factor being applied to eachprediction block may be ⅓. When four prediction blocks are generated,the weighting factor being applied to each prediction block may be ¼.Alternatively, the final prediction block of the current block may bedetermined by applying different weighting factors to prediction blocks.

The weighting factor does not have to have a fixed value for eachprediction block, and may have a variable value for each predictionblock. Here, the weighting factors being applied to prediction blocksmay be the same, and may be different from each other. For example, whentwo prediction blocks are generated, weighting factors being applied tothe two prediction blocks may be a variable value for each block such as(½, ½), (⅓, ⅔), (¼, ¾), (⅖, ⅗), (⅜, ⅝), etc. In the meantime, theweighting factor may be a positive real number and a negative realnumber. For example, the weighting factor may be a negative real numbersuch as (−½, 3/2), (−⅓, 4/3), (−¼, 5/4), etc.

In the meantime, in order to apply a variable weighting factor, at leastone piece of weighting factor information for the current block may besignaled through a bitstream. The weighting factor information may besignaled for each prediction block, and may be signaled for eachreference picture. A plurality of prediction blocks may share one pieceof weighting factor information.

The encoder and the decoder may determine whether or not motioninformation of the merge candidate is used based on a prediction blocklist utilization flag. For example, when a prediction block listutilization flag indicates one, which is a first value, for eachreference picture list, it may indicate that the encoder and the decodercan use motion information of the merge candidate of the current blockto perform inter prediction or motion compensation. When a predictionblock list utilization flag indicates zero which is a second value, itmay indicate that the encoder and the decoder do not perform interprediction or motion compensation by using motion information of themerge candidate of the current block. In the meantime, the first valueof the prediction block list utilization flag may be set to zero, andthe second value thereof may be set to one.

The following formula 3 to formula 5 show examples of generating thefinal prediction block of the current block, when the inter-predictionindicator of each current block is PRED_BI (or when the current blockcan use two pieces of motion information), PRED_TRI (or when the currentblock can use three pieces of motion information), and PRED_QUAD (orwhen the current block can use four pieces of motion information) andprediction direction for each reference picture list is uni-direction.

P_BI=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+RF)>>1  [Formula 3]

P_TRI=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+RF)/3  [Formula4]

P_TRI=(WF_L0*P_L0+OFFSET_L0+WF_L1*P_L1+OFFSET_L1+WF_L2*P_L2+OFFSET_L2+WF_L3*P_L3+OFFSET_L3+RF)>>2  [Formula5]

In formulas 3 to 5, the final prediction block of the current block maybe designated as P_BI, P_TRI, and P_QUAD, and the reference picture listmay be designated as LX (X=0, 1, 2, 3). A weighting factor value of theprediction block generated by using LX may be designated as WF_LX. Anoffset value of the prediction block generated by using LX may bedesignated as OFFSET_LX. A prediction block generated by using motioninformation for LX of the current block may be designated as P_LX. Arounding factor may be designated as RF, and may be set to zero, apositive number, or a negative number. The LX reference picture list mayinclude at least one of a long-term reference picture, a referencepicture on which deblocking filter is not performed, a reference pictureon which sample adaptive offset is not performed, a reference picture onwhich adaptive loop filter is not performed, a reference picture onwhich deblocking filter and adaptive offset are performed, a referencepicture on which deblocking filter and adaptive loop filter isperformed, a reference picture on which sample adaptive offset andadaptive loop filter are performed, a reference picture on whichdeblocking filter, sample adaptive offset, and adaptive loop filter areperformed. In this case, the LX reference picture list may be at leastone of an L0 reference picture list, an L1 reference picture list, an L2reference picture list, and an L3 reference picture list.

When the prediction direction for a predetermined reference picture listis a plurality of directions, the final prediction block for the currentblock may be obtained based on the weighted sum of the predictionblocks. Here, weighting factors being applied to prediction blocksderived from the same reference picture list may have the same value,and may have different values.

At least one of a weighting factor (WF_LX) and an offset (OFFSET_LX) fora plurality of prediction blocks may be a coding parameter being entropyencoded/decoded.

As another example, a weighting factor and an offset may be derived froman encoded/decoded neighbor block adjacent to the current block. Here,the neighbor block of the current block may include at least one of ablock being used in deriving the spatial merge candidate of the currentblock and a block being used in deriving the temporal merge candidate ofthe current block.

As another example, a weighting factor and an offset may be determinedbased on a display order (POC) of the current picture and referencepictures. In this case, when the current picture is far from thereference picture, the weighting factor or the offset may be set to asmall value. When the current picture is close to the reference picture,the weighting factor or the offset may be set to a large value. Forexample, when the POC difference between the current picture and L0reference picture is two, the weighting factor value being applied tothe prediction block generated by referring to L0 reference picture maybe set to ⅓. In contrast, when the POC difference between the currentpicture and L0 reference picture is one, the weighting factor valuebeing applied to the prediction block generated by referring to L0reference picture may be set to ⅔. As described above, the weightingfactor or offset value may be in inverse proportion to the display orderdifference between the current picture and the reference picture. Asanother example, the weighting factor or offset value may be inproportion to the display order difference between the current pictureand the reference picture.

As another example, based on at least one of coding parameters of thecurrent block, the neighbor block, and the collocated block, at leastone of a weighting factor and an offset may be entropy encoded/decoded.Also, based on at least one of coding parameters of the current block,the neighbor block, and the collocated block, the weighted sum ofprediction blocks may be calculated.

The weighted sum of a plurality of prediction blocks may be applied toonly a partial area in the prediction block. Here, the partial area maybe an area corresponding to the boundary in the prediction block. Asdescribed above, in order to apply the weighted sum to only the partialarea, the weighted sum may be applied for each sub-block of theprediction block.

Next, the entropy encoding/decoding of information on motioncompensation at steps S1204 and S1301 will be described in detail.

FIGS. 30 and 31 are views showing examples of syntax of information onmotion compensation. FIG. 30 shows an example of syntax of informationon motion compensation in a coding unit (coding_unit). FIG. 31 shows anexample of syntax of information on motion compensation in a predictionunit (prediction_unit).

The encoding apparatus may entropy encode information on motioncompensation through a bitstream, and the decoding apparatus may entropydecode information on motion compensation included in the bitstream.Here, information on motion compensation being entropy encoded/decodedmay include at least one of information on whether or not a skip mode isused (cu_skip_flag), information on whether or not a merge mode is used(merge_flag), merge index information (merge_index), an inter-predictionindicator (inter_pred_idc), a weighting factor value (wf_I0, wf_I1,wf_I2, wf_I3), and an offset value (offset_I0, offset_I1, offset_I2,offset_I3). Information on motion compensation may be entropyencoded/decoded in at least one of a CTU, a coding block, and aprediction block.

When information on whether or not a skip mode is used (cu_skip_flag)has one which is a first value, it may indicate the use of the skipmode. When it has two which is a second value, it may not indicate theuse of the skip mode. Motion compensation of the current block may beperformed by using the skip mode based on the information on whether ornot a skip mode is used.

When information on whether or not a merge mode is used (merge_flag) hasone which is a first value, it may indicate the use of the merge mode.When it has zero which is a second value, it may not indicate the use ofthe merge mode. Motion compensation of the current block may beperformed by using the merge mode based on the information on whether ornot a merge mode is used.

The merge index information (merge_index) may mean informationindicating a merge candidate in the merge candidate list.

Also, the merge index information may mean information on a merge index.

Also, the merge index information may indicate a block, which derived amerge candidate, among reconstructed blocks spatially/temporallyadjacent to the current block.

Also, the merge index information may indicate at least one piece ofmotion information of the merge candidate. For example, when the mergeindex information has zero that is a first value, it may indicate thefirst merge candidate in the merge candidate list. When it has one thatis a second value, it may indicate the second merge candidate in themerge candidate list. When it has two that is a third value, it mayindicate the third merge candidate in the merge candidate list. In thesame manner, when it has a fourth value to N-th value, it may indicate amerge candidate corresponding to the value according to the order in themerge candidate list. Here, N may mean a positive integer includingzero.

Motion compensation of the current block may be performed by using themerge mode based on merge mode index information.

When the current block is encoded/decoded in inter prediction, theinter-prediction indicator may mean at least one of inter predictiondirection and the number of prediction directions of the current block.For example, the inter-prediction indicator may indicate uni-directionalprediction, or multi-directional prediction such as bi-directionalprediction, tri-directional prediction, quad-directional prediction,etc. The inter-prediction indicator may mean the number of referencepictures being used when the current block generates the predictionblock. Alternatively, one reference picture may be used formulti-directional prediction. In this case, M reference pictures areused to perform N(N>M)-directional prediction. The inter-predictionindicator may mean the number of prediction blocks being used inperforming motion compensation or inter prediction for the currentblock.

As described above, according to the inter-prediction indicator, thenumber of reference pictures being used in generating the predictionblock of the current block, the number of prediction blocks being usedin performing inter prediction or motion compensation of the currentblock, or the number of reference picture lists available to the currentblock, etc. may be determined. Here, the number of reference picturelists is N that is a positive integer such as 1, 2, 3, 4 or more. Forexample, the reference picture list may include L0 L1, L2, L3, etc.Motion compensation may be performed on the current block by using atleast one reference picture list.

For example, the current block may generate at least one predictionblock by using at least one reference picture list to perform motioncompensation of the current block. For example, one or more predictionblocks may be generated by using reference picture list L0 to performmotion compensation. Alternatively one or more prediction blocks may begenerated by using reference picture lists L0 and L1 to perform motioncompensation. Alternatively, one or more prediction block or at most Nprediction blocks (here, N is a positive integer equal to or greaterthan three or two) may be generated by using reference picture lists L0L1 and L2 to perform motion compensation. Alternatively, one or moreprediction blocks or at most N prediction blocks (here, N is a positiveinteger equal to or greater than four or two) may be generated by usingreference picture lists L0, L1, L2 and L3 to perform motion compensationof the current block.

The reference picture indicator may indicate uni-direction (PRED_LX),bi-direction (PRED_BI), tri-direction (PRED_TRI), quad-direction(PRED_QUAD) or greater direction depending on the number of predictiondirections of the current block.

For example, when uni-directional prediction is performed for eachreference picture list, the inter-prediction indicator PRED_LX may meanthat one prediction block is generated by using a reference picture listLX (X is an integer such as 0, 1, 2, or 3, etc.) and inter prediction ormotion compensation is performed by using the generated one predictionblock. The inter-prediction indicator PRED_BI may mean that twoprediction blocks are generated by using at least one of L0 L1, L2 andL3 reference picture lists and inter prediction or motion compensationis performed by using the generated two prediction blocks. Theinter-prediction indicator PRED_TRI may mean that three predictionblocks are generated by using at least one of L0 L1, L2, and L3reference picture lists and inter prediction or motion compensation isperformed by using the generated three prediction blocks. Theinter-prediction indicator PRED_QUAD may mean that four predictionblocks are generated by using at least one of L0 L1, L2, and L3reference picture lists and inter prediction or motion compensation isperformed by using the generated four prediction blocks. That is, thesum of the number of prediction blocks used in performing interprediction of the current block may be set to the inter-predictionindicator.

When performing multi-directional prediction for the reference picturelist, the inter-prediction indicator PRED_BI may mean performingbi-directional prediction for the L0 reference picture list. Theinter-prediction indicator PRED_TRI may mean: performing tri-directionalprediction for the L0 reference picture list; performing uni-directionalprediction for the L0 reference picture list, and performingbi-directional prediction for the L1 reference picture list; orperforming bi-directional prediction for the L0 reference picture listand performing uni-directional prediction for the L1 reference picturelist.

As described above, the inter-prediction indicator may mean that atleast one to at most N (here, N is the number of prediction directionsindicated by the inter-prediction indicator) prediction blocks aregenerated from at least one reference picture list so as to performmotion compensation. Alternatively, the inter-prediction indicator maymean that at least one to at most N prediction blocks are generated fromN reference pictures and motion compensation for the current block isperformed by using the generated prediction block.

For example, the inter-prediction indicator PRED_TRI may mean that threeprediction blocks are generated by using at least one of L0 L1, L2, andL3 reference picture lists so as to perform inter prediction or motioncompensation of the current block. Alternatively, the inter-predictionindicator PRED_TRI may mean that three prediction blocks are generatedby using at least three selected from the group consisting of L0 L1, L2,and L3 reference picture lists so as to perform inter prediction ormotion compensation of the current block. Also, the inter-predictionindicator PRED_QUAD may mean that four prediction blocks are generatedby using at least one of L0 L1, L2, and L3 reference picture lists so asto perform inter prediction or motion compensation of the current block.Alternatively, the inter-prediction indicator PRED_QUAD may mean thatfour prediction blocks are generated by using at least four selectedfrom the group consisting of L0 L1, L2, and L3 reference picture listsso as to perform inter prediction or motion compensation of the currentblock.

Available inter prediction directions may be determined according to theinter-prediction indicator, and all or some of the availableinter-prediction directions may be selectively used based on the sizeand/or shape of the current block.

A prediction list utilization flag indicates whether or not theprediction block is generated by using the reference picture list.

For example, when the prediction list utilization flag indicates onewhich is a first value, it may indicate that the prediction block isgenerated by using the reference picture list. When it indicates zerowhich is a second value, it may indicate that the prediction block isnot generated by using the reference picture list. Here, the first valueof the prediction list utilization flag may be set to zero, and thesecond value thereof may be set to one.

That is, when the prediction list utilization flag indicates the firstvalue, the prediction block of the current block may be generated byusing motion information corresponding to the reference picture list.

In the meantime, the prediction list utilization flag may be set basedon the inter-prediction indicator. For example, when theinter-prediction indicator indicates PRED_LX, PRED_BI, PRED_TRI orPRED_QUAD, the prediction list utilization flag predFlagLX may be set toone which is a first value. When the inter-prediction indicator isPRED_LN (N is a positive integer other than X), the prediction listutilization flag predFlagLX may be set to zero which is a second value.

Also, the inter-prediction indicator may be set based on the predictionlist utilization flag. For example, when the prediction list utilizationflag predFlagL0 and the predFlagL1 indicate one which is a first value,the inter-prediction indicator may be set to PRED_BI. For example, whenonly the prediction list utilization flag predFlagL0 indicate one whichis a first value, the inter-prediction indicator may be set to PRED_L0.

When two or more prediction blocks are generated during motioncompensation for the current block, the final prediction block for thecurrent block may be generated through a weighted sum for eachprediction block. When calculating the weighted sum, at least one of aweighting factor and an offset may be applied for each prediction block.A weighted sum factor such as the weighting factor or the offset, etc.used in calculating the weighted sum may be entropy encoded/decoded forat least one of a reference picture list, a reference picture, a motionvector candidate index, a motion vector difference, a motion vector,information on whether or not a skip mode is used, information onwhether or not a merge mode is used, merge index information. Also, theweighted sum factor of each prediction block may be entropyencoded/decoded based on the inter-prediction indicator. Here, theweighted sum factor may include at least one of the weighting factor andthe offset.

The weighted sum factor may be derived by index information specifyingone of predetermined sets in the encoding apparatus and the decodingapparatus. In this case, index information for specifying at least oneof the weighting factor and the offset may be entropy encoded/decoded.The predetermined set in the encoder and the decoder may be respectivelydefined for a weighting factor and an offset. The predetermined set mayinclude at least one weighting factor candidate or at least one offsetcandidate. Alternatively, a table defining the mapping relation betweenthe weighting factor and the offset may be used. In this case, theweighting factor value and the offset value for the prediction block maybe obtained from the table by using one piece of index information.Index information on the offset mapped to each piece of indexinformation for the weighting factor being entropy encoded/decoded maybe entropy encoded/decoded.

Information related to the weighted sum factor may be entropyencoded/decoded by a block unit, and may be entropy encoded/decoded at ahigher level. For example, the weighting factor or the offset may beentropy encoded/decoded by a block unit such as a CTU, a CU, or a PU,etc., or may be entropy encoded/decoded at a higher level such as avideo parameter set, a sequence parameter set, a picture parameter set,an adaptation parameter set, or a slice header, etc.

The weighted sum factor may be entropy encoded/decoded based on aweighted sum factor difference value between a weighted sum factor and aweighted sum factor prediction value. For example, a weighting factorprediction value and a weighting factor difference value may be entropyencoded/decoded, or an offset prediction value and an offset differencevalue may be entropy encoded/decoded. Here, the weighting factordifference value may indicate a difference value between the weightingfactor and the weighting factor prediction value, and the offsetdifference value may indicate a difference value between the offset andthe offset prediction value.

Here, the weighted sum factor difference value may be entropyencoded/decoded by a block unit, and the weighted sum factor predictionvalue may be entropy encoded/decoded at a higher level. When theweighted sum factor prediction value such as the weighting factorprediction value or the offset prediction value, etc. is entropyencoded/decoded by a picture or a slice unit, blocks included in thepicture or the slice may use a common weighted sum factor predictionvalue.

The weighted sum factor prediction value may be derived through aparticular area within an image, a slice or a tile or through aparticular area within a CTU or a CU. For example, the weighting factorvalue or the offset value of the particular area within an image, aslice, a tile, a CTU, or a CU may be used as the weighting factorprediction value or the offset prediction value. In this case, entropyencoding/decoding of the weighted sum factor prediction value may beomitted, and entropy encoding/decoding of only the weighted sum factordifference value may be performed.

Alternatively, the weighted sum factor prediction value may be derivedfrom an encoded/decoded neighbor block adjacent to the current block.For example, the weighting factor value or the offset value of theencoded/decoded neighbor block adjacent to the current block may be setto the weighting factor prediction value or the offset prediction valueof the current block. Here, the neighbor block of the current block mayinclude at least one of a block used in deriving the spatial mergecandidate and a block used in deriving the temporal merge candidate.

When using the weighting factor prediction value and the weightingfactor difference value, the decoding apparatus may calculate aweighting factor value for the prediction block by adding the weightingfactor prediction value and the weighting factor difference value. Also,when using the offset prediction value and the offset difference value,the decoding apparatus may calculate an offset value for the predictionblock by adding the offset prediction value and the offset differencevalue.

The weighted sum factor or the weighted sum factor difference value maybe entropy encoded/decoded based on at least one of coding parameters ofthe current block, the neighbor block, and the collocated block.

Based on at least one of coding parameters of the current block, theneighbor block, and the collocated block, the weighted sum factor, theweighted sum factor prediction value, or the weighted sum factordifference value may be derived as a weighted sum factor, a weighted sumfactor prediction value, or a weighted sum factor difference value ofthe current block.

Instead of entropy encoding/decoding information on a weighted sumfactor of the current block, a weighted sum factor of an encoded/decodedblock adjacent to the current block may be used as a weighted sum factorof the current block. For example, the weighting factor or the offset ofthe current block may be set to have the same value as the weightingfactor or the offset of the encoded/decoded neighbor block adjacent tothe current block.

Motion compensation may be performed on the current block by using atleast one of weighted sum factors, or may be performed by using at leastone of derived weighted sum factors.

The weighted sum factor may be included in information on motioncompensation.

At least one piece of the above-described information on motioncompensation may be entropy encoded/decoded in at least one of a CTU anda sub-unit of a CTU (sub-CTU). Here, the sub-unit of the CTU may includeat least one of a CU and a PU. A block of the sub-unit of the CTU mayhave a square shape or a non-square shape. Information on motioncompensation may mean at least one piece of information on motioncompensation for convenience.

When information on motion compensation is entropy encoded/decoded inthe CTU, motion compensation may be performed on all or a part of blocksexisting in the CUT by using information on the motion compensationaccording to a value of information on motion compensation.

When information on motion compensation is entropy encoded/decoded inthe CTU or the sub-unit of the CTU, information on motion compensationmay be entropy encoded/decoded based on at least one of the size and thedepth of the predetermined block.

Here, information on the size or the depth of the predetermined blockmay be entropy encoded/decoded. Alternatively, information on the sizeor the depth of the predetermined block may be determined based on atleast one of a preset value in the encoder and the decoder and a codingparameter or based at least one of another syntax element values.

Information on motion compensation may be entropy encoded/decoded inonly a block that is larger than or equal to the predetermined block insize, and information on motion compensation may not be entropyencoded/decoded in a block that is smaller than the predetermined blockin size. In this case, motion compensation may be performed onsub-blocks in a block that is larger than or equal to the predeterminedblock in size based on information on motion compensation that isentropy encoded/decoded in a block that is larger than or equal to thepredetermined block in size. That is, sub-blocks in a block that islarger than or equal to the predetermined block in size may shareinformation on motion compensation including a motion vector candidate,a motion vector candidate list, a merge candidate, a merge candidatelist, etc.

Information on motion compensation may be entropy encoded/decoded inonly a block that is shallower than or equal to the predetermined blockin depth, and information on motion compensation may not be entropyencoded/decoded in a block that is deeper than the predetermined blockin depth. In this case, motion compensation may be performed onsub-blocks in a block that is shallower than or equal to thepredetermined block in depth based on information on motion compensationentropy encoded/decoded in a block that is shallower than or equal tothe predetermined block in depth. That is, sub-blocks in a block that isshallower than or equal to the predetermined block in depth may shareinformation on motion compensation including a motion vector candidate,a motion vector candidate list, a merge candidate, a merge candidatelist, etc.

For example, when information on motion compensation is entropyencoded/decoded in the sub-unit, of which the size is 32×32, of the CTU,of which the block size is 64×64, motion compensation may be performedon a block that is included in a 32×32-size block and is smaller thanthe 32×32-size block in size based on information on motion compensationthat is entropy encoded/decoded in the 32×32-size block.

As another example, when information on motion compensation is entropyencoded/decoded in the sub-unit, of which the size is 16×16, of the CTU,of which the block size is 128×128, motion compensation may be performedon a block that is included in a 16×16-size block and is smaller than orequal to the 16×16-size block in size based on information on motioncompensation that is entropy encoded/decoded in the 16×16-size block.

As another example, when information on motion compensation is entropyencoded/decoded in the sub-unit, of which the block depth is one, of theCTU, of which the block depth is zero, motion compensation may beperformed on a block that is included in a one-depth block and is deeperthan the one-depth block in depth based on information on motioncompensation that is entropy encoded/decoded in the one-depth block.

For example, when at least one piece of information on motioncompensation is entropy encoded/decoded in the sub-unit, of which theblock depth is two, of the CTU, of which the block depth is zero, motioncompensation may be performed on a block that is included in a two-depthblock and is equal to or is deeper than the two-depth block in depthbased on information on motion compensation that is entropyencoded/decoded in the two-depth block.

Here, the value of the block depth may be a positive integer includingzero. When the value of the block depth is large, it may mean that thedepth is deep. When the value of the block depth is small, it may meanthat the depth is shallow. Accordingly, when the value of the blockdepth is large, the block size may be small. When the value of the blockdepth is small, the block size may be large. Also, a sub-block of apredetermined block may be deeper than the predetermined block in depth,and a sub-block of a predetermined block may be deeper than thepredetermined block in depth within a block corresponding to thepredetermined block.

Information on motion compensation may be entropy encoded/decoded foreach block, and may be entropy encoded/decoded at a higher level. Forexample, information on motion compensation may be entropyencoded/decoded for each block such as a CTU, a CU, or a PU, etc., ormay be entropy encoded/decoded at a higher lever such as a VideoParameter Set, a Sequence Parameter Set, a Picture Parameter Set, anAdaptation Parameter Set or a slice Header, etc.

Information on motion compensation may be entropy encoded/decoded basedon an information difference value on motion compensation indicating adifference value between information on motion compensation and aninformation prediction value on motion compensation. Considering theinter-prediction indicator which is one piece of information on motioncompensation as an example, the inter-prediction indicator predictionvalue and the inter-prediction indicator difference value may be entropyencoded/decoded. Here, the inter-prediction indicator difference valuemay be entropy encoded/decoded for each block, and the inter-predictionindicator prediction value may be entropy encoded/decoded at a higherlevel. The information prediction value on motion compensation such asthe inter-prediction indicator prediction value, etc. is entropyencoded/decoded for each picture or slice, blocks in the picture or theslice may use a common information prediction value on motioncompensation.

The information prediction value on motion compensation may be derivedthrough a particular area within an image, a slice or a tile, or througha particular area within a CTU or a CU. For example, theinter-prediction indicator of the particular area in the image, slice,tile, CTU, or CU may be used as the inter-prediction indicatorprediction value. In this case, entropy encoding/decoding of theinformation prediction value on motion compensation may be omitted, andonly the information difference value on motion compensation may beentropy encoded/decoded.

Alternatively, the information prediction value on motion compensationmay be derived from an encoded/decoded neighbor block adjacent to thecurrent block. For example, the inter-prediction indicator of theencoded/decoded neighbor block adjacent to the current block may be setto the inter-prediction indicator prediction value of the current block.Here, the neighbor block of the current block may include at least oneof a block used in deriving the spatial merge candidate and a block usedin deriving the temporal merge candidate. Also, the neighbor block mayhave the same depth as or the smaller depth than that of the currentblock. When there is a plurality of neighbor blocks, one may beselectively used according to the predetermined priority. The neighborblock used to predict information on motion compensation may have afixed position based on the current block, and may have a variableposition according to the position of the current block. Here, theposition of the current block may be a position based on a picture or aslice including the current block, or may be a position based on aposition of a CTU, a CU, or a PU including the current block.

The merge index information may be calculated by using index informationin the predetermined sets in the encoder and the decoder.

When using the information prediction value on motion compensation andthe information difference value on motion compensation, the decodingapparatus may calculate an information value on motion compensation forthe prediction block by adding the information prediction value onmotion compensation and the information difference value on motioncompensation.

Information on motion compensation or the information difference valueon motion compensation may be entropy encoded/decoded based on at leastone of coding parameters of the current block, the neighbor block, andthe collocated block.

Based on at least one of coding parameters of the current block, theneighbor block, and the collocated block, information on motioncompensation, information prediction value on motion compensation, orinformation difference value on motion compensation may be derived asinformation on motion compensation, information prediction value onmotion compensation or information difference value on motioncompensation of the current block.

Instead of entropy encoding/decoding information on motion compensationof the current block, information on motion compensation of theencoded/decoded block adjacent to the current block may be used asinformation on motion compensation of the current block. For example,the inter-prediction indicator of the current block may be set to thesame value as the inter-prediction indicator of the encoded/decodedneighbor block adjacent to the current block.

Also, at least one piece of information on motion compensation may havea fixed value preset in the encoder and the decoder. The preset fixedvalue may be determined as a value for at least one piece of informationon motion compensation. Blocks which are smaller than a particular blockin size within the particular block may share at least one piece ofinformation on motion compensation having the preset fixed value. In thesame manner, blocks which are deeper than a particular block in depthand are sub-blocks of the particular block may share at least one pieceof information on motion compensation having the preset fixed value.Here, the fixed value may be a positive integer including zero, or maybe an integer vector value including (0, 0).

Here, sharing at least one piece of information on motion compensationmay mean that blocks have the same value for at least one piece ofinformation on motion compensation, or that motion compensation isperformed on the blocks by using the same value for at least one pieceof information on motion compensation.

Information on motion compensation may further include at least one of amotion vector, a motion vector candidate, a motion vector candidateindex, a motion vector difference value, a motion vector predictionvalue, information on whether or not a skip mode is used (skip_flag),information on whether or not a merge mode is used (merge_flag), mergeindex information (merge_index), motion vector resolution information,overlapped block motion compensation information, local illuminationcompensation information, affine motion compensation information,decoder-side motion vector derivation information, and bi-directionaloptical flow information.

Motion vector resolution information may be information indicatingwhether or not particular resolution is used for at least one of amotion vector and a motion vector difference value. Here, resolution maymean precision. Also, particular resolution may set to at least one ofan integer-pixel (integer-pel) unit, a ½-pixel (½-pel) unit, a ¼-pixel(¼-pel) unit, a ⅛-pixel (⅛-pel) unit, a 1/16-pixel ( 1/16-pel) unit, a1/32-pixel ( 1/32-pel) unit, and a 1/64-pixel ( 1/64-pel) unit.

Overlapped block motion compensation information may be informationindicating whether or not the weighted sum of the prediction block ofthe current block is calculated by using a motion vector of the neighborblock spatially adjacent to the current block during motion compensationof the current block.

Local illumination compensation information may be informationindicating whether or not at least one of a weighting factor value andan offset value is applied when generating the prediction block of thecurrent block. Here, at least one of the weighting factor value and theoffset value may be a value calculated based on a reference block.

Affine motion compensation information may be information indicatingwhether or not an affine motion model is used during motion compensationof the current block. Here, the affine motion model may be a model forpartitioning one block into several sub-blocks by using a plurality ofparameters, and calculating motion vectors of the partitioned sub-blocksby using representative motion vectors.

Decoder-side motion vector derivation information may be informationindicating whether or not a motion vector required for motioncompensation is used by being derived by the decoder. Information on amotion vector may not be entropy encoded/decoded based on decoder-sidemotion vector derivation information. Also, when decoder-side motionvector derivation information indicates that the decoder derives anduses a motion vector, information on a merge mode may be entropyencoded/decoded. That is, decoder-side motion vector derivationinformation may indicate whether or not a merge mode is used in thedecoder.

Bi-directional optical flow information may be information indicatingwhether or not motion compensation is performed by correcting a motionvector for each pixel or sub-block. Based on bi-directional optical flowinformation, a motion vector for each pixel or sub-block may not beentropy encoded/decoded. Here, motion vector correction may be modifyinga motion vector value from a motion vector for each block into a motionvector for each pixel or sub-block.

Motion compensation is performed on the current block by using at leastone piece of information on motion compensation, and at least one pieceof information on motion compensation may be entropy encoded/decoded.

FIG. 32 is a view showing an embodiment of using a merge mode in blocks,which are smaller than a predetermined block in size, of a CTU.

Referring to FIG. 32 , when the size of the predetermined block is 8×8,blocks smaller than 8×8 (slashed blocks) may use the merge mode.

In the meantime, when comparing the sizes between blocks, being smallerthan the predetermined block in size may mean that the sum of samples inthe block is small. For example, a 32×16-size block may have 512samples, and thus the 32×16-size block is smaller than a 32×32-sizeblock having 1024 samples. A 4×16-size block may have 64 samples, andthus the 4×16-size block may be equal to a 8×8 block in size.

When entropy encoding/decoding information on motion compensation, abinarization method such as a truncated rice binarization method, a K-thorder Exp_Golomb binarization method, a limited K-th order Exp_Golombbinarization method, a fixed-length binarization method, a unarybinarization method, or a truncated unary binarization method, etc. maybe used.

When entropy encoding/decoding information on motion compensation, acontext model may be determined by using at least one piece ofinformation on motion compensation of a neighbor block adjacent to thecurrent block or area information of the neighbor block, information onpreviously encoded/decoded motion compensation or previouslyencoded/decoded area information, information on the depth of thecurrent block, and information on the size of the current block.

Also, when entropy encoding/decoding information on motion compensation,entropy encoding/decoding may be performed by using at least one pieceof information on motion compensation of the neighbor block, informationon previously encoded/decoded motion compensation, information on thedepth of the current block, and the information on the size of thecurrent block as a prediction value for information on motioncompensation of the current block.

The encoding/decoding process may be performed for each of luma andchroma signals. For example, in the encoding/decoding process, at leastone method of obtaining an inter-prediction indicator, generating amerge candidate list, deriving motion information, and performing motioncompensation may be differently applied for a luma signal and a chromasignal.

The encoding/decoding process may be equally performed for luma andchroma signals. For example, in the encoding/decoding process beingapplied for the luma signal, at least one of an inter-predictionindicator, a merge candidate list, a merge candidate, a referencepicture, and a reference picture list may be equally applied to thechroma signal.

The methods may be performed in the encoder and the decoder in the samemanner. For example, in the encoding/decoding process, at least onemethod of obtaining an inter-prediction indicator, generating a mergecandidate list, deriving motion information, and performing motioncompensation may be applied to the encoder and the decoder equally.Also, orders of applying the methods may be different in the encoder andthe decoder.

The embodiments of the present invention may be applied according to thesize of at least one of the coding block, the prediction block, theblock, and the unit. Here, the size may be defined as the minimum sizeand/or the maximum size in order to apply the embodiments, and may bedefined as a fixed size to which the embodiment is applied. Also, afirst embodiment may be applied in a first size, and a second embodimentmay be applied in a second size. That is, the embodiments may bemultiply applied according to the size. Also, the embodiments of thepresent invention may be applied only when the size is equal to orgreater than the minimum size and is equal to or less than the maximumsize. That is, the embodiments may be applied only when the block sizeis in a predetermined range.

For example, only when the size of the encoding/decoding target block isequal to or greater than 8×8, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block isequal to or greater than 16×16, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block isequal to or greater than 32×32, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block isequal to or greater than 64×64, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block isequal to or greater than 128×128, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block is4×4, the embodiments may be applied. For example, only when the size ofthe encoding/decoding target block is equal to or less than 8×8, theembodiments may be applied. For example, only when the size of theencoding/decoding target block is equal to or less than 16×16, theembodiments may be applied. For example, only when the size of theencoding/decoding target block is equal to or greater than 8×8 and isequal to or less than 16×16, the embodiments may be applied. Forexample, only when the size of the encoding/decoding target block isequal to or greater than 16×16 and is equal to or less than 64×64, theembodiments may be applied.

The embodiments of the present invention may be applied according to atemporal layer. An identifier for identifying the temporal layer towhich the embodiment can be applied may be signaled, and the embodimentsmay be applied for the temporal layer specified by the identifier. Here,the identifier may be defined as indicating the minimum layer and/or themaximum layer to which the embodiment can be applied, and may be definedas indicating a particular layer to which the embodiment can be applied.

For example, only when the temporal layer of the current picture is thelowest layer, the embodiments may be applied. For example, only when atemporal layer identifier of the current picture is zero, theembodiments may be applied. For example, only when the temporal layeridentifier of the current picture is equal to or greater than one, theembodiments may be applied. For example, only when the temporal layer ofthe current picture is the highest layer, the embodiments may beapplied.

As described in the embodiment of the present invention, a referencepicture set used in processes of reference picture list construction andreference picture list modification may use at least one of referencepicture lists L0 L1, L2, and L3.

According to the embodiments of the present invention, when a deblockingfilter calculates boundary strength, at least one to at most N motionvectors of the encoding/decoding target block may be used. Here, Nindicates a positive integer equal to or greater than 1 such as 2, 3, 4,etc.

In motion vector prediction, when the motion vector has at least one ofa 16-pixel (16-pel) unit, a 8-pixel (8-pel) unit, a 4-pixel (4-pel)unit, an integer-pixel (integer-pel) unit, a ½-pixel (½-pel) unit, a¼-pixel (¼-pel) unit, a ⅛-pixel (⅛-pel) unit, a 1/16-pixel ( 1/16-pel)unit, a 1/32-pixel ( 1/32-pel) unit, and a 1/64-pixel ( 1/64-pel) unit,the embodiments of the present invention may be applied. Also, inperforming the merge mode, the motion vector may be optionally used foreach pixel unit.

A slice type to which the embodiments of the present invention may bedefined and the embodiments of the present invention may be appliedaccording to the slice type.

For example, when the slice type is a T (Tri-predictive)-slice, aprediction block may be generated by using at least three motionvectors, and may be used as the final prediction block of theencoding/decoding target block by calculating a weighted sum of at leastthree prediction blocks. For example, when the slice type is a Q(Quad-predictive)-slice, a prediction block may be generated by using atleast four motion vectors, and may be used as the final prediction blockof the encoding/decoding target block by calculating a weighted sum ofat least four prediction blocks.

The embodiment of the present invention may be applied to interprediction and motion compensation methods using the merge mode, as wellas inter prediction and motion compensation methods using motion vectorprediction, and inter prediction and motion compensation methods usingthe skip mode, etc.

The shape of the block to which the embodiments of the present inventionis applied may have a square shape or a non-square shape.

A method for encoding and decoding a video by using a merge modeaccording to the present invention have been described above withreference to FIGS. 12 to 32 . Hereinafter, a method for decoding avideo, a method for encoding a video, an apparatus for decoding a video,an apparatus for encoding a video, and a bitstream according to thepresent invention will be described in detail with reference to FIGS. 33and 34 .

FIG. 33 is a view showing a method for decoding a video according to thepresent invention.

Referring to FIG. 33 , a merge candidate list of a current blockincluding at least one merge candidate corresponding to each of aplurality of reference picture lists may be generated at step S3301.

Here, the merge candidate corresponding to each of the plurality ofreference picture lists may mean a merge candidate having LX motioninformation corresponding to reference picture list LX. For example,there are L0 merge candidate having L0 motion information, L1 mergecandidate having L1 motion information, L2 merge candidate having L2motion information, L3 merge candidate having L3 motion information,etc.

In the meantime, the merge candidate list includes at least one of aspatial merge candidate derived from a spatial neighbor block of thecurrent block, a temporal merge candidate derived from a collocatedblock of the current block, a modified spatial merge candidate derivedby modifying the spatial merge candidate, a modified temporal mergecandidate derived by modifying the temporal merge candidate, and a mergecandidate having a predefined motion information value. Here, the mergecandidate having the predefined motion information value may be a zeromerge candidate.

In this case, the spatial merge candidate may be derived from asub-block of a neighbor block adjacent to the current block. Also, thetemporal merge candidate may be derived from a sub-block of thecollocated block of the current block.

In the meantime, the merge candidate list may further include a combinedmerge candidate derived by using at least two selected from the groupconsisting of the spatial merge candidate, the temporal merge candidate,the modified spatial merge candidate, and the modified temporal mergecandidate.

Also, at least one piece of motion information may be determined byusing the generated merge candidate list at step S3302.

Also, the prediction block of the current block may be generated byusing the determined at least one piece of motion information at stepS3303.

Here, the generating of the prediction block of the current block atstep S3303 may include: generating a plurality of temporary predictionblocks according to an inter-prediction indicator of the current block;and generating the prediction block of the current block by applying atleast one of a weighting factor and an offset to the generated pluralityof temporary prediction blocks.

In this case, at least one of the weighting factor and the offset may beshared in blocks that are smaller than a predetermined block in size orare deeper than the predetermined block in depth.

In the meantime, the merge candidate list may be shared in blocks thatare smaller than a predetermined block in size or are deeper than thepredetermined block in depth.

Also, when the current block is smaller than a predetermined block insize or is deeper than the predetermined block in depth, the mergecandidate list may be generated based on a high block of the currentblock. The higher bock is equal to the predetermined block in size or indepth.

The prediction block of the current block may be generated by applyinginformation on a weighted sum to a plurality of prediction blocksgenerated based on a plurality of merge candidates or a plurality ofmerge candidate lists.

FIG. 34 is a view showing a method for encoding a video according to thepresent invention.

Referring to FIG. 34 , a merge candidate list of a current blockincluding at least one merge candidate corresponding to each of aplurality of reference picture lists may be generated at step S3401.

At least one piece of motion information may be determined by using thegenerated merge candidate list at step S3402.

Also, a prediction block of the current block may be generated by usingthe determined at least one piece of motion information at step S3403.

An apparatus for decoding a video according to the present invention mayinclude an inter prediction unit that generates a merge candidate listof a current block including at least one merge candidate correspondingto each of a plurality of reference picture lists, determines at leastone piece of motion information by using the merge candidate list, andgenerates a prediction block of the current block by using thedetermined at least one piece of motion information.

An apparatus for encoding a video according to the present invention mayinclude an inter prediction unit that generates a merge candidate listof a current block including at least one merge candidate correspondingto each of a plurality of reference picture lists, determines at leastone piece of motion information by using the merge candidate list, andgenerates a prediction block of the current block by using thedetermined at least one piece of motion information.

A bitstream according to the present invention may be a bitstreamgenerated by a method for encoding a video, the method including:generating a merge candidate list of a current block including at leastone merge candidate corresponding to each of a plurality of referencepicture lists; determining at least one piece of motion information byusing the merge candidate list; and generating a prediction block of thecurrent block by using the determined at least one piece of motioninformation.

In the above-described embodiments, the methods are described based onthe flowcharts with a series of steps or units, but the presentinvention is not limited to the order of the steps, and rather, somesteps may be performed simultaneously or in different order with othersteps. In addition, it should be appreciated by one of ordinary skill inthe art that the steps in the flowcharts do not exclude each other andthat other steps may be added to the flowcharts or some of the steps maybe deleted from the flowcharts without influencing the scope of thepresent invention.

The embodiments include various aspects of examples. All possiblecombinations for various aspects may not be described, but those skilledin the art will be able to recognize different combinations.Accordingly, the present invention may include all replacements,modifications, and changes within the scope of the claims.

The embodiments of the present invention may be implemented in a form ofprogram instructions, which are executable by various computercomponents, and recorded in a computer-readable recording medium. Thecomputer-readable recording medium may include stand-alone or acombination of program instructions, data files, data structures, etc.The program instructions recorded in the computer-readable recordingmedium may be specially designed and constructed for the presentinvention, or well-known to a person of ordinary skilled in computersoftware technology field. Examples of the computer-readable recordingmedium include magnetic recording media such as hard disks, floppydisks, and magnetic tapes; optical data storage media such as CD-ROMs orDVD-ROMs; magneto-optimum media such as floptical disks; and hardwaredevices, such as read-only memory (ROM), random-access memory (RAM),flash memory, etc., which are particularly structured to store andimplement the program instruction. Examples of the program instructionsinclude not only a mechanical language code formatted by a compiler butalso a high level language code that may be implemented by a computerusing an interpreter. The hardware devices may be configured to beoperated by one or more software modules or vice versa to conduct theprocesses according to the present invention.

Although the present invention has been described in terms of specificitems such as detailed elements as well as the limited embodiments andthe drawings, they are only provided to help more general understandingof the invention, and the present invention is not limited to the aboveembodiments. It will be appreciated by those skilled in the art to whichthe present invention pertains that various modifications and changesmay be made from the above description.

Therefore, the spirit of the present invention shall not be limited tothe above-described embodiments, and the entire scope of the appendedclaims and their equivalents will fall within the scope and spirit ofthe invention.

INDUSTRIAL APPLICABILITY

The present invention may be used in an apparatus for encoding/decodingan image.

What is claimed is:
 1. A method for decoding a video, the methodcomprising: generating a merge candidate list for a current block;deriving first motion information for first reference picture list andsecond motion information for second reference picture list from themerge candidate list of the current block; and refining the first motioninformation and the second motion information based on a distortioncalculated based on sum of absolute differences (SAD).
 2. The method ofclaim 1, wherein a prediction direction of the first reference picturelist is different from a prediction direction of the second referencepicture list.
 3. The method of claim 1, wherein the refined first motioninformation and the refined second motion information are derived insub-block units of the current block.
 4. A method for encoding a video,the method comprising: generating a merge candidate list for a currentblock; deriving first motion information for first reference picturelist and second motion information for second reference picture listfrom the merge candidate list of the current block; and refining thefirst motion information and the second motion information based on adistortion calculated based on sum of absolute differences (SAD).
 5. Anon-transitory computer readable medium storing a bitstream formed by amethod for encoding a video, wherein the method comprises: generating amerge candidate list for a current block; deriving first motioninformation for first reference picture list and second motioninformation for second reference picture list from the merge candidatelist of the current block; and refining the first motion information andthe second motion information based on a distortion calculated based onsum of absolute differences (SAD).